Voting open for your top 3 tracks in our Odd Meter Remix/ReMangle/Reimagine Challenge

Prizes generously donated by:
ADDAC System (Eurorack 3U)
Endorphin.es (Eurorack 3U)
Fred's Lab (DIY Paraphonic Synth)
Northern Light Modular (Buchla-format h-series OR Eurorack3U)
Rattly and Raw (Wiggler Approved Kontakt Instrument)
Studio.h (Buchla-format 4U)

Get your vote on and free 42 minute modular playlist for your listening pleasure.

windows programming for video synthesis?

Discussion of modular and standalone video generating/processing techniques and associated hardware.

Moderators: Kent, luketeaford, Joe., daverj

Post Reply
mercyofwolves
Common Wiggler
Posts: 124
Joined: Fri Sep 10, 2021 1:06 pm
Location: messupatameear

windows programming for video synthesis?

Post by mercyofwolves » Mon Nov 15, 2021 9:05 am

i've done some video art programming for win32 using GDI. my system was limited to OpenGL 1.4 which was history by the time i got there, so never picked up DirectX or OpenGL, so i'm used to using GetPixel, SetPixel and gradient triangles, and i'm not used to whatever everybody else is doing.

now i've started Direct2D on a modern windows system which is definately a significant performance improvement by processor speed alone. Direct2D doesn't seem to have an explicit SetPixel operation (for Device Independent Pixels or otherwise) which looks like a real impediment without resorting back to GDI methods (depending on what can be done with custom effects, which i haven't had a go at yet) so that i can perform arbitrary creative operations on the bitmap data. the whole idea of "using small rectangles or ellipses" as effective pixels doesn't appeal to me, but i have absolutely no experience with graphics acceleration so maybe that's normal. since i've never been able to buy the most modern computers i've never been interested in the performance differences between technology options but more upon what i can do with elementary functionality. i use c++ on what i have, i couldn't tell you a thing about different types of processors, sdks, libraries for doing things, anything.

the DirectX addon for visual studio is almost 9 gigs and would about max out my hard drive before i've done anything with it, so not an option.

i'm quite used to finding ways to do things with the GDI and c++ without making use of acceleration but given a faster system it might be worthwhile to see what i can make happen. aside from 9 gig additions to visual studio, what sensible options are there to improved video processing? i'm not interested in developing for other OS or really much more than making a functional app, and tbh much of what i do is sound data visualisation so using geometry driven architecture against point driven architecture is a bit silly. if i want to make arbitrary operations on a video file, stream, or screen, are Direct2D effects extensive enough, are GDI/GDI+ methods more appropriate for processing, does Open GL offer much over Direct2D in this respect? am i cheating myself until i potentiate DirectX? always i am trying to learn as little as possible about other peoples' methods since that's the enjoyable/development part, i don't enjoy learning much so try to minimise it when possible. also using 3rd party resources for any native function isn't something i can comply with.

Marizu
Common Wiggler
Posts: 119
Joined: Tue Sep 27, 2016 5:12 pm

Re: windows programming for video synthesis?

Post by Marizu » Wed Nov 24, 2021 5:02 am

These days SDK's are large but they do provide a lot of functionality that makes the development much quicker/easier.

You haven't really explained what spec of system you have or what type of visuals you aspire to producing. That makes it difficult to provide any specific advice.

As a general rule, with modern graphics development, you try to offload as much work as possible on to the GPU. This is because the GPU is a massively powerful parallel processor. I think that my GPU has about 700 processing cores, as opposed to the 8 cores in my CPU. I can throw massive amounts of unoptimised geometry at it and it doesn't miss a beat. This allows me to concentrate on a creative outcome as opposed to wasting time optimising a data structure in order to increase performance.

You could look at Processing, which is a Java based framework for creative programming. I don't know how big the download for that is.
You could also look to learning shader programming in GLSL. That would require no download as you can do it in a browser (www.shadertoy.com , https://thebookofshaders.com). The GLSL language has a C based syntax but has a lot of limitations and differences because it is optimised for mass parallelisation.

I'm not massively experienced in this area, but I have done some stuff with Unity (massive download) and I have a basic understanding of shader programming.

mercyofwolves
Common Wiggler
Posts: 124
Joined: Fri Sep 10, 2021 1:06 pm
Location: messupatameear

Re: windows programming for video synthesis?

Post by mercyofwolves » Thu Nov 25, 2021 9:17 am

i think it's kind of a dud question produced by the usual issue of having options limited by finance. trying to discern how i can access pixel specific operations with my limitations, i think the answer is going to be to do as much as possible with what is possible. i've been following inigo quilez for a decade without being able to run shadertoy because of an opengl 1.4 limit :p this laptop was $210 new this year, it's taken several years to acquire a 64 bit OS system.

mercyofwolves
Common Wiggler
Posts: 124
Joined: Fri Sep 10, 2021 1:06 pm
Location: messupatameear

Re: windows programming for video synthesis?

Post by mercyofwolves » Fri Dec 03, 2021 8:48 am

looking over the direct2d effects yesterday,
there's a 3x3 matrix transform which ought to be capable of some good stuff (polar to cartesian mapping)

and i briefly looked over the "custom effect", but not enough to understand the limits of the method,

but going through all the direct2d effects, i can't see anything to mirror the screen, even orthogonally, let alone at an arbitrary axis, only mirror the brush, which i'm not sure can be taken from the preexisting screen..

mercyofwolves
Common Wiggler
Posts: 124
Joined: Fri Sep 10, 2021 1:06 pm
Location: messupatameear

Re: windows programming for video synthesis?

Post by mercyofwolves » Sat Dec 04, 2021 8:15 am

can't find a lot of material on using direct2d effects, the process seems rather inscrutible, not a single tutorial on applying an effect in direct2d.

found a funny video from microsoft, it's the first one you'll find at the top of youtube for using effects in direct2d.

the first 53 minutes of the video is a blank podium waiting for someone to begin.

the only comment on the video is someone laughing that, at 53 minutes, the presenter says, "i think we can begin,"

so, i wouldn't want to give the impression from my posts that direct2d effects are anything that are meant to be used by the public.

there is a stackexchange question where someone recognises that the direct2d effect implementation isn't set up for application to the rendering image...... imported bitmaps yes, geometric primitives yes (can't figure out the context tag tho) but definately not an effect like blur or anything on the actual rendering region.

the demo does show that it has possibilities for motile remapping, distortion, eg. video synthesis, but the guy's voice is quavering like a little girl probably because he knows he's presenting an API that's been hamstringed by inscrutible implementation (eg. the documents on d2d do not list the headers you need... d2d1effects.h is listed, but not the requirement for d2d1_1.h instead of d2d1.h, or the dxguid.lib none of this will compile without, only mentioned on stackexchange.)

Post Reply

Return to “Video Synthesis”