1
\$\begingroup\$

Perhaps some noob questions for a scientific application that you can assist with: I am trying to render a large, evenly spaced grid of data in 2D. The full data set dimensions could be anywhere from 48x250,000 to 1000x15,000,000 points/pixels.

The data is evenly spaced, but I need to change the aspect ratio, and then render only a section of that data on the screen at a time. I need the users to be able to smoothly scroll forward or backward along the long dimension with user input, or press the "play" button and have it move forward or backward at certain speed.

I am looking for advice on how to handle this via DirecX or OpenGL (probably using SharpDX or OpenTK).

The basic approach I am considering is to use textures that the application generates on the fly from the data and then map them to a very simple model. I would keep the graphics card supplied with a large enough scene so that as the user scrolls or moves around that they don't have to wait for the application. Questions I have are:

1) does this approach make sense? i am presuming this is way more efficient and faster than creating a bunch of primitives with discrete colors. true?

2) is there a particular texture format that is good to use and easy to create (or do i create it directly)? I have the data in application memory as large arrays and it loads/unloads from disk as needed. I would map the scalar values to a color as I create the image/texture blocks.

3) would mipmaps be useful? i could have an aspect ratio (or small enough window) that there are fewer screen pixels than data in the viewable range. Or, would it just automatically subsample?

4) if mipmaps are used, most of the scaling is along the long dimension, so could i uses non-square subsamples?

4) does creating compressed images make sense? or, would the compression overhead yield much benefit (or how might i determine this)

5) how would i go about finding an optimal texture size?

Any other general advice?

Hopefully this is a straightforward problem for the cool stuff y'all do in games!

|improve this question
\$\endgroup\$

closed as too broad by Anko, congusbongus, bummzack, Josh♦ Oct 10 '14 at 17:02

Please edit the question to limit it to a specific problem with enough detail to identify an adequate answer. Avoid asking multiple distinct questions at once. See the How to Ask page for help clarifying this question. If this question can be reworded to fit the rules in the help center, please edit the question.

  • \$\begingroup\$ So what you have now is a set of data which can be interpreted as a large 2D grid of some values. Now what you want to present is a part of this data (in a single frame) where the two screen dimensions represent the position in your data array and the color corresponds to the actual value of a specific element in your array? Also are you implementing the zoom as well or is one pixel representing one data sample? \$\endgroup\$ – Matic Oblak Oct 9 '14 at 15:54
  • \$\begingroup\$ Exactly. We do need to implement a zoom--both keeping the aspect ratio as well as "zooming"/scaling along one axis while keeping the other constant. (There is also some basic "picking" involved to get the coordinates of a point and drill back into our application data, but I am presuming that is relatively trivial). \$\endgroup\$ – jrowe88 Oct 9 '14 at 16:20
  • 1
    \$\begingroup\$ If you are new to graphics I would highly suggest using XNA 4.0 for this, as the library already has plenty of functionality to solve most of your problems easily. That includes: creating textures from datasets, visualizing textures (with very simple and intuitive (code-vise) scrolling/scaling). You would have to make a clever algorithm for converting a 1000x15000000 datatable into a texture, especially if the data dynamically changes more than once per second, but other than that - XNA will do everything for you. And YES this approach is better than rendering a billion polygons on the screen. \$\endgroup\$ – cubrman Oct 9 '14 at 17:21
  • \$\begingroup\$ What have you tried and why is it not working as well/efficiently/prettily as you wanted? \$\endgroup\$ – Anko Oct 9 '14 at 20:38
  • \$\begingroup\$ Hello Anko, we were using VTK, which is a higher level api on OpenGL. It seems to be more designed for meshes and for our "simpler" situation, we don't have much control over how it uses textures. It is also cross platform and some compromises are unappealing--e.g., fonts are horrendous \$\endgroup\$ – jrowe88 Oct 15 '14 at 13:53
1
\$\begingroup\$

  1. Yes, this is a fine approach - it will be orders of magnitude faster than creating a primitive per data point.

  2. Assuming single-channel data, you should use whatever DXGI_FORMAT (or equivalent in OGL) maps to your source data, and convert in the pixel shader. There are native types for 8, 16, and 32-bit integers, as well as 32-bit floats. You should also only create a few textures up front and then reuse them - don't create them on the fly. Instead, keep track of which ones are off-screen and overwrite the data of e.g. the least-recently-used one.

  3. If you allow scaling out significantly, MIP maps will be useful. Be sure to enable 16x anisotropic filtering though, since you'll have a huge uv-delta mismatch basically all the time. 4(first): Anisotropic filtering does precisely this.

  4. I wouldn't bother with GPU compressed formats initially. Your bottleneck will almost certainly be disk IO, so if anything I would use something like LZMA to compress the chunks of data on disk.

  5. A good size would be to use a height of whatever your data height is (48 - 1000), and use 1024 or 2048 for your width. This should offer a good balance of texture count versus total memory used (and the ability to swap out LRU texture data).

Other tips:

  1. You might intially think to create a geometry and camera that you conceptually scan accross it. A much simpler approach is to create a full-screen quad, and just update the texture coordinates on the fly using a cbuffer with scale/offset parameters.

  2. Do all your visualization math (color gradients, etc.) in the pixel shader - don't convert on upload unless it's required to fit into one of the hardware native formats (e.g. if you're using a custom number format for some reason). Sub-point: if your data is double-precision, use the R32G32 format, then in the shader use the asdouble(color.r, color.g) instruction to get the original value back. Note that double-precision support is an optional feature though - not all hardware supports it.

|improve this answer
\$\endgroup\$

Not the answer you're looking for? Browse other questions tagged or ask your own question.