You may need to add dropout layer to avoid overfit

No, the data already comes in the “correct” format as a data stream from a sensor. The DNN models are trained to work on that data stream, on an edge device you usually don’t have the performance to pre process anything, except for an FFT which is a fixed function accelerator that can process the data in real time.

Haha that's amazing to hear because I had a similar experience too! The data scientist on the team was re-initializing the model, the image matrices, and other objects in the data pipeline over and over again for every inference. I had to decouple those in a similar way as you did.

Everyone usually thinks the model is the problem but I am starting to think that the rest of the code is usually what actually needs to be optimized

Deepstream is a framework that provides an efficient pipeline for taking video input, run detector and tracker and extract the output. (its capable of much more..its just a simple description). Take a look at the demo programs available in the deepstream installation folder..or in deepstream python api github page.

I have used it mainly for implementing object detection related activities.

Well, I needed to solve my problem so I looked at papers / software solutions. Then it was a slow process of many iterations of trials and errors.

I couldn't tell you what would be the best for your use case tho. I am afraid it's been too long for me to remember the details. Beside, each method may be more or less effective according to good results on the metrics/inference time or the method and means of training that you can afford.

I can give you a tip : I initialized my inference script only once per boot, and then put it in "waiting mode" so I wouldn't have to initialize the model for each inference (it's the largest cause of losing time). Then upon receiving a socket message, the script would read a data file, do an inference pass, write the results in an another file, delete/move the data to storage and wait for the next socket message. It's obvious when you think about it that you absolutely don't want to call/initialize your inference script once per inference, but, well, you never know what people think about :p

Oh wow I never heard about Deepstream in my earlier project. Have you used it? Were you able to bring it on mid-project or did you have to use it from the start of the project?

Cool, how did you learn all those techniques? And how did you determine which one was the major cause of too much memory usage / too slow inference time?

Interesting, hadn't heard of TVM before! I'm wondering, did you come across cases in your work where it wasn't the model that was the worst bottleneck but the pre-processing / data pipeline that actually needed to be optimized? I had one experience like that so just wondering how common it is

You seem to be asking for the purposes of photogrammetry right?

I presume it could be scaled to 3D easily from 2D

But also I don’t know much and am stupid

NVIDIA provides deepstream framework for optimisation. Also you can convert your model from FP32 to INT8 and improve the speed (albeit sacrificing some accuracy. Have to figure out the trade off). deepstream is available in c++ while python APIs are also available to implement your model the way you see fit.

I had to deploy my models on nvidia jetson / tx for my last job, 1.5y ago.

In these use cases there is a lot of optimisation to do. A list of available methods: pruning, mixed precision training/inference, quantization, CUDA/onnx/nvidia optimization, training models that perform on lower resolution data via knowledge distillation from models that trained on higher res data...

Look around, this is on the top of my head from a bit of time ago. There is plenty of resources now for inference on the edge.

I contribute to our group who is working exactly on this. Currently, it is quite a mess because each HW vendor provides its own tooling for deploying on their device which leads to a lot of problems (e.g. missing support for certain layers). One of the most promising tools for edge deployment is TVM together with Network Architecture Search (NAS) where the network is tailored for a specific use case and the available resources.

Isn't this just Voronoi interpolation?

I see, was a bit confused from the title of what your goal was

How do you choose which points to connect?

Not sure I understand what you mean by depth.

I used FuncAnimation in matplotlib. I made it a feature of this package to visualize the process of "densifying" a given point cloud.

This is a tool with deep learning applications, not a tool that uses deep learning.

You're correct, but that's not what he's doing.

I feel like deep learning is overengineering this problem

Nice work! How did you create this video?

Thankss

Yes. In order to ensure the contextual token embedding attends to longer text, you need to use a model that accepts larger sequence lengths. Check out Longformer

Yes, i know but in this way the embedding of a word will be created according only to the tokens of the sentence in which it is found, right?

You're supposed to pass in each sentence separately, as a list of sentences. You do not pass all the sentences as one string.