1
\$\begingroup\$

I am trying to simulate what's known as an 'event camera' in Unreal. Unlike conventional cameras that output full frames, event cameras return the locations of pixels where activity is detected - see example below.

enter image description here

I have a post-process material in UE, whose job is to essentially subtracts the current image from the previous one at every tick and 'light up' all the pixels that have changed. Thus, the job of this material to identify all "moving" pixels, and after doing this subtraction, I can identify all the pixel locations that had a change in intensity.

Under the assumption that the number of changed pixels is much lesser than the total number of pixels, I want to avoid the really expensive 'ReadPixels' sort of operations in order to simulate the output of an event camera. Is it possible to access the UV positions of non-zero pixels directly? What is a good, efficient way to access such data and read it out from a material?

\$\endgroup\$
6
  • \$\begingroup\$ What do you need to do with this data? \$\endgroup\$ – DMGregory Feb 22 '20 at 4:54
  • \$\begingroup\$ I am trying to simulate what's known as an 'event camera' in Unreal. Unlike conventional cameras that output full frames, event cameras return the locations of pixels where activity is detected: which are in this case, the non-zero pixel locations \$\endgroup\$ – HighVoltage Feb 22 '20 at 5:28
  • \$\begingroup\$ Does that have to be read back to the CPU as a list, or can you work with it on the GPU side as a mask or somesuch, for the next step to use? \$\endgroup\$ – DMGregory Feb 22 '20 at 5:31
  • \$\begingroup\$ Ideally, I would like to read it back to the CPU so it can be streamed/written to disk as an array or something like that. I am not sure how to handle this data purely on the GPU's side yet, but that's an interesting thought from an efficiency point of view. \$\endgroup\$ – HighVoltage Feb 22 '20 at 5:40
  • 1
    \$\begingroup\$ If CPU readback is unavoidable, then you might need to use a compute shader to gather up the changed pixels and append then to one or more shortlists to read back. \$\endgroup\$ – DMGregory Mar 2 '20 at 12:28

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.