Reintegrate projectM Visualizer

jonas

But that code is using ImGui?

I actually got some presets from the original pack (https://github.com/projectM-visualizer/presets-projectm-classic) to work with ProjectM version 4 now using Qt 5 and QGLWidget. The ones starting with "Martin", but a few others too. No crashes either.
With Qt 6 and QOpenGLWidget, none of them work, and I also see crashes.

Gustavo L Conte

@jonas I did the version stuff in the initializeGL, tested with qLog if it was setting the version, it did, and it didnt change anything.....

I'm thinking now, maybe these crashes could be some sort of related to overhead? It would be a shot in the dark again, but maybe increase the intervals? in the timer formulae

Gustavo L Conte

@jonas said in Reintegrate projectM Visualizer:

@Gustavo-L-Conte

I actually got some presets from the original pack (https://github.com/projectM-visualizer/presets-projectm-classic) to work with ProjectM version 4 now using Qt 5 and QGLWidget. The ones starting with "Martin", but a few others too. No crashes either.
With Qt 6 and QOpenGLWidget, none of them work, and I also see crashes.

tomorrow I'm going to clone the classic presets... if the ones I was using arent already, I got from steam and/or ubuntu packages;

I was able to make qt6 work with v3 today, remember? I believe using visualizations2 branch.
Had to force QStringLiterals all over the place to make it run, but perphaps with your fixes today it wont be necessary anymore. It was mainly paths: fontfile, preset, texture, etc;

Its good try to test with the same suite of presets, obviously.

Its very nice that you advanced making v4 work at all! I need to rest, but tomorrow I'm going to test both visualizations latest and visualizations2 if necessary.

tkssssss nice job

Screenshot from 2024-06-30 00-56-06.png
qt6 llibprojectM v3 with visualizations2 branch

I found another "hint" for the puzzle, on the qt frontend of projectM repo

void resizeGL ( int w, int h ) override
		{
			// Setup viewport, projection etc
			setup_opengl ( w,h );
			projectm_set_window_size(m_projectM->instance(), static_cast<size_t>(w), static_cast<size_t>(h));
		}
		
		void setup_opengl ( int w, int h )
		{

			/* Our shading model--Gouraud (smooth). */
			glShadeModel ( GL_SMOOTH );
			/* Culling. */
			//    glCullFace( GL_BACK );
			//    glFrontFace( GL_CCW );
			//    glEnable( GL_CULL_FACE );
			/* Set the clear color. */
			glClearColor ( 0, 0, 0, 0 );
			/* Setup our viewport. */
			glViewport ( 0, 0, w, h );
			/*
					* Change to the projection matrix and set
					* our viewing volume.
			*/
			glMatrixMode ( GL_TEXTURE );
			glLoadIdentity();

			//    gluOrtho2D(0.0, (GLfloat) width, 0.0, (GLfloat) height);
			glMatrixMode ( GL_PROJECTION );
			glLoadIdentity();

			//    glFrustum(0.0, height, 0.0,width,10,40);
			glMatrixMode ( GL_MODELVIEW );
			glLoadIdentity();

			glDrawBuffer ( GL_BACK );
			glReadBuffer ( GL_BACK );
			glEnable ( GL_BLEND );

			glBlendFunc ( GL_SRC_ALPHA, GL_ONE_MINUS_SRC_ALPHA );
			// glBlendFunc(GL_SRC_ALPHA, GL_ONE);
			glEnable ( GL_LINE_SMOOTH );
			glEnable ( GL_POINT_SMOOTH );
			glClearColor ( 0.0f, 0.0f, 0.0f, 0.0f );
//   glClear(GL_COLOR_BUFFER_BIT);

			// glCopyTexImage2D(GL_TEXTURE_2D,0,GL_RGB,0,0,texsize,texsize,0);
			//glCopyTexSubImage2D(GL_TEXTURE_2D,0,0,0,0,0,texsize,texsize);
			glLineStipple ( 2, 0xAAAA );
		}

https://github.com/projectM-visualizer/frontend-qt/blob/master/src/common/qprojectmwidget.hpp

Hope it helps; we can use QOpenGLFunctions.h for that procedure. Maybe this is just whats missing for us now in terms of black/blank screen............................

Gustavo L Conte

@Gustavo-L-Conte said in Reintegrate projectM Visualizer:

/* Our shading model--Gouraud (smooth). /
glShadeModel ( GL_SMOOTH );
/ Culling. /
// glCullFace( GL_BACK );
// glFrontFace( GL_CCW );
// glEnable( GL_CULL_FACE );
/ Set the clear color. /
glClearColor ( 0, 0, 0, 0 );
/ Setup our viewport. /
glViewport ( 0, 0, w, h );
/
* Change to the projection matrix and set
* our viewing volume.
*/
glMatrixMode ( GL_TEXTURE );
glLoadIdentity();
  	//    gluOrtho2D(0.0, (GLfloat) width, 0.0, (GLfloat) height);
  	glMatrixMode ( GL_PROJECTION );
  	glLoadIdentity();

  	//    glFrustum(0.0, height, 0.0,width,10,40);
  	glMatrixMode ( GL_MODELVIEW );
  	glLoadIdentity();

  	glDrawBuffer ( GL_BACK );
  	glReadBuffer ( GL_BACK );
  	glEnable ( GL_BLEND );

  	glBlendFunc ( GL_SRC_ALPHA, GL_ONE_MINUS_SRC_ALPHA );
  	// glBlendFunc(GL_SRC_ALPHA, GL_ONE);
  	glEnable ( GL_LINE_SMOOTH );
  	glEnable ( GL_POINT_SMOOTH );
  	glClearColor ( 0.0f, 0.0f, 0.0f, 0.0f );
// glClear(GL_COLOR_BUFFER_BIT);
  	// glCopyTexImage2D(GL_TEXTURE_2D,0,GL_RGB,0,0,texsize,texsize,0);
  	//glCopyTexSubImage2D(GL_TEXTURE_2D,0,0,0,0,0,texsize,texsize);
  	glLineStipple ( 2, 0xAAAA );

after making lotsa tests with this idea of "preparing" opengl I've decided to debug the preset loading, it seems something is wrong with the "current" preset, it never starts. I did some serious qLog debugging messages, it appears that it loads the presets correctly but the playlist never starts, not even the M logo. tomorrow I'll try to understand this better....

jonas

@Gustavo-L-Conte

I've made a much more minimal test project here for debugging:
https://github.com/jonaski/ProjectMTest

Gustavo L Conte

@jonas said in Reintegrate projectM Visualizer:

@Gustavo-L-Conte

I've made a much more minimal test project here for debugging:
https://github.com/jonaski/ProjectMTest

Awesome idea! Without v4, I always get these errors both in Strawberry and also now in the testproject:

~/ProjectMTest/build$ ./projectmtest 
No Textures Loaded from "/usr"/share/projectM/textures
Could not open font file:

What I'm doing to bypass it is kinda bizarre.

mkdir -p \"/usr\"/share/projectM

for the first error, then I copy stuff to that bizarre new local dir.

For the fonts, I just force the filename to some ttf in /fonts with QStringLiteral and .toStdString(); in the settings s structure, vars are menuFontURL and titleFontURL

I really do not understand where this double quotes come from, neither why I'm having to set the fonts filename by hand. Do you think maybe v3 is trying to read from a config file i do not have?

Gustavo L Conte

its hard-coded on my lib from Ubuntu package

I'll see if it fixes by recompiling it by hand

~/ProjectMTest/build$ strings /usr/lib/libprojectM.so | xargs -0 | grep \"/usr

Smooth Preset DuHard Cut SensitiAspect CorrectioEaster Egg ParamSoft Cut Ratingsidle://Geiss & Sperl - Feedback (projectM idle Hle HDR mix).milkSmooth Transitionsition Duration

"/usr"/share/proprojectM/presets "/usr/share/fonts/truetype/ttf-dejavu/DejaVuSansaVuSansMono.ttf"

edit: I'm so stupid, the ubuntu package is v2 not v3 LoL just realized it now

Gustavo L Conte

Okey, got libprojectM v2.10 and v3.12 from sourceforge and...

Looks like v 3.12 works like a charm in the test program. Didn't need to do anything, out-of-the-box. Some presets seem "idle" maybe because they are not fed by any data? there is no ConsumeBuffer right?

about v2.10 compiled by hand: it gave some trouble to compile, had to fix some stuff in the source for it to compile,also fiddle with src/CMakeLists.txt to avoid trying to detect things we dont need in this context. Amazing:

~/ProjectMTest/build$ ./projectmtest 
No Textures Loaded from "/usr"/share/projectM/textures
Could not open font file:

Something is really messed up with that v2.10, not just in Ubuntu now I see

~/ProjectMTest/build$ ls -la /usr/lib/libprojectM.so.2
lrwxrwxrwx 1 root root 20 jun 30 21:29 /usr/lib/libprojectM.so.2 -> libprojectM.so.2.1.0
~/ProjectMTest/build$ date
dom 30 jun 2024 21:31:32 -03
~/ProjectMTest/build$ strings /usr/lib/libprojectM.so.2 | grep usr\/share
"/usr/share/projectM/fonts/Vera.ttf"
"/usr/share/projectM/fonts/VeraMono.ttf"
~/ProjectMTest/build$ strings /usr/lib/libprojectM.so.2 | grep usr\"\/share
"/usr"/share/projectM/presets
"/usr"/share/projectM/textures

edit: Found the reason!
in v2.10 It defines the path here,
line 279 src/libprojectM/Renderer/TextureManager.cpp

void TextureManager::loadTextureDir()
{
        std::string dirname = CMAKE_INSTALL_PREFIX  "/share/projectM/textures";

the CMAKE_INSTALL_PREFIX is being set with the double quotes, probably some REGEX replace on cmake

Gustavo L Conte

I'm testing libprojectM v3.12 with qt6 and visualizations2 branch, seems to work pretty good!

I'm using the cream of the cream presets pack on their repo. Some are probably breaking indeed. But no segfaults!!!

but i think its not consuming the buffer.... most presets doesnt seem to interact with the song

Gustavo L Conte

Because of suspecting the presets are not interacting with the song, I did code a few lines that change the behaviour of instead use ConsumeBuffer from gstreamer, to acquire data from the monitor pulse/pipewire source of the audio card; it then sends this recorded buffer to projectM. honestly i did not notice much difference, this can be a little subjective, but maybe it does work. If any one is interested I can put the code somewhere..

Gustavo L Conte

  if (projectm_) {
    int samples_per_channel = map.size / sizeof(int) / 2; 
    qDebug() << "Samples per channel:" << samples_per_channel;
    const float *data = reinterpret_cast<float*>(map.data);
    qDebug() << "First 10 data samples:" << QVector<float>(data, data + 10);
    projectm_->pcm()->addPCMfloat_2ch(data, samples_per_channel);
  }

I'm messing around here, but maybe I've made some progress in the issue of the buffer not being properly consumed. The basic idea here is prevent samples per channel to become negative, in my point of view this should never happen. I also changed to the float version of the AddPCM function. Still, not all presets seem to work, but i.e. "Geiss - Drop Shadow 1.milk" shows that its interacting with the music. It's very important to focus in the presets that give information, thats one I recommend. I also recommend the ones that make a waveform, they are the best for debuging.

I used to see a triangle kinda half the screen but after these changes the presets that are broken render a white screen. LoL

I'm using v3.12 projectM, qt6 with visualizations2 branch. The other branch is very CPU intensive, maybe because of resets (if you turn resets on in visualization2 same high load cpu happens), dunno if this happens with NVIDIA.

Gotta rest, cya guys

Gustavo L Conte

Here's a very simple milkdrop preset to test and compare running it inside Strawberry ( libprojectM v3.12 + Qt6 ) against the projectMSDL official visualizer. The results are pretty disappointing, since on the SDL app, it renders as expected, while on Strawberry the screen just flashes (at least the right color)

[settings]
presetAuthor=Gustavo L  Conte
presetName=000 Simplest Waveform for Strawberry Music Player debug

[init]
# Initialization code here
nWaveMode=2  # Use the waveform mode 2, which is lines
wave_r=0.0
wave_g=1.0
wave_b=0.0
wave_a=1.0
nWaveDots=0
nWaveThick=1
bAdditiveWaves=0

[per_frame_1]
# Per-frame equations
wave_r = 0.0;
wave_g = 1.0;
wave_b = 0.0;
wave_a = 1.0;

[waveform]
# Waveform equations
x = sample;  # Directly map the sample values to the x-coordinate
y = value1;  # Use the waveform's y-coordinate as given
red = 0.0;
green = 1.0;
blue = 0.0;
alpha = 1.0;

Gustavo L Conte

I think i'm on the right track for the issues of my previous flood. I'm sorry for so many messages, but each one had an important milestone maybe and I wanted to report as soon as possible.

These tests now were made with latest visualisations branch, libprojectM version less < than 4, of course. I kinda gave up fixing v4 by now. And Qt6, therefore QOpenGLWidget. I can put this code somewhere but dunno if anybody is interested, its not too big anyway. I believe it can really help debug due to the facts I've mentioned earlier.

Not consuming the buffer
Made again lotsa tests, turns out that the ConsumeBuffer is not functioning at all. With the help of the pulse code that consumes the pulse/pipewire monitor sink, that I mentioned earlier, It could be determined that it works this way, you can see interaction of the preset output and the music, but not with the Clementine code. I'm pretty sure Of this now. I recommend the waveform presets like "Zylot - Crosshair Dimensions (Light of the Ages).milk" to test.
Delay of almost 5 seconds in output
When I could see the music interacting with the output, I've noticed there is a 5 sec delay between the visualization and the beats, etc; this is easily noticed by stopping the music and seeing the waveform stop only 5 seconds later, the contrary obviously happens too.
very Bad performance (at least on my Intel board)
If I comment out the reset function in the drawing method, performance is nice again, otherwise, it consumes 100% of a whole cpu core.
OpenGL issues and deformed output
I've inserted some gl codes and looks like it fixes almost every single plugin output.

I think the one that really fixes is this, on drawBackground

// Set the viewport to match the scene dimensions
    glViewport(0, 0, static_cast<GLsizei>(sceneRect().width() * pixel_ratio_), static_cast<GLsizei>(sceneRect().height() * pixel_ratio_));

also those maybe necessary: (put on the resize window method)

// Adjust the viewport and projection
    glViewport(0, 0, width(), height());

    // Set up the projection matrix
    glMatrixMode(GL_PROJECTION);
    glLoadIdentity();
    glOrtho(0, width(), height(), 0, -1, 1); // Adjust as needed for your coordinate system

    // Switch back to model-view matrix
    glMatrixMode(GL_MODELVIEW);
    glLoadIdentity();

I believe this helps a lot, but I kinda think its not clearing the window as it should. Some plugins make a "trail" in the screen with indicates that.

// Clear the background to avoid flickering
   glClear(GL_COLOR_BUFFER_BIT | GL_DEPTH_BUFFER_BIT);

I put that on the drawing method but maybe its the wrong way IN Qt.

and even still, something is missing, because even if it fixes the majority of plugins, the simplest waveform test I made earlier is still broken.

edit> forgot to mention, I do have an hypothesis to fix the ConsumeBuffer.

The DATA arrives as a char, while add PCM expects an int. I think the reinterpret_cast does not solve this, it only makes us paint a huge wall with a pencil, we need a brush!!

Gustavo L Conte

I've managed to fix the latency of 5 seconds, it was just because of buffer sizes in the pulseaudio capturing code. Now its instant! whee

Soounds like the ResetGL is important too. by putting it back, several presets started to work better including the Simplest waveform milkdrop test. But the "perspective" is different from what the official projectMSDL renders. At least now it isnt just a flashlight...

Gustavo L Conte

WEll, disabling resetGL makes most of the plugins work exactly like projectMSDL
also performance is decent
I saw on the frontend code that they only use resetGL to INIT and when resizing.

Trying hard here to understand the projection matrix, trying both with orthogonal and gluPerspective

But there are those plugins that brake without resetGL, like the simplest waveform test and some others that turn into a white screen. Anyway even those that work only with resetGL on the render loop, they don't render exactly like they should, and most plugins renders real bad.

jonas

@Gustavo-L-Conte
I suspect this might be a Qt bug, or some issue with using projectm and Qt together, although I'm not experienced with OpenGL at all, and I simply lack time to dig more into it since I already have to much work to do with Strawberry. I also tried QOpenGLWindow like the projectm maintainer suggested, but I'm getting similar results.
https://github.com/orgs/projectM-visualizer/discussions/820#discussioncomment-9911992

Gustavo L Conte

i'm trying now to make ConsumeBuffer work as expected, the pulseaudio Capture code helped me a lot to ddebug the presets because some doest not work with none activity.
I've tested forcing opengl profiles via MEsa envvars, it changes the behaviour of some plugins. But there are a few that really wont work and gives just a white screen. I'm comparing to the real projectMSDL and seveeral are functioning ok, it is certainly a bug in way Qt handles the projection view matrix with QOpenGLWidget etc.. or something related to context too.

export MESA_GLSL_VERSION_OVERRIDE=130; 
export MESA_GL_VERSION_OVERRIDE="4.6";

disable resetGL you will see it works much better, several plugins work equal to the original
if we fix ConsumeBuffer, which is not feeding projectM, and remove resetGL, most plugins will work and music will output the visualization properly. I think this bug or whatever it is someday will come to light and then its just a matter of fixing the implementation. There are many other stuff to care about in the application, I agree. This should not go to master, I believe. Its better to wait ppl from llibprojectM -- Maybe if in v4 they find the way with Qt, then it would solve everything, including these version-ballet between v2 v3 v4 etc

Gustavo L Conte

@jonas here's a present for you

if (projectm_) {
  const int samples_per_channel = static_cast<int>(map.size) / sizeof(int) / 4;
  const float *data = reinterpret_cast<float*>(map.data);
  projectm_->pcm()->addPCMfloat_2ch(data, samples_per_channel);
}

Not rdy yet, but on the face of the goal

progressssssssssssssssssssssssssssssssssssssszzzzzzzzz

If i divide by two only half the waveform are rendered, so I tried dividing by 4
Gonna understand that, in the mean time, if you're still going to work on it, change the ConsumeBuffer rooutine ASAP so things work for better debugging....

Gustavo L Conte

this works too, even better, and makes much more sense

const unsigned int samples_per_channel = static_cast<unsigned int>(map.size / sizeof(size_t) / 2);
const float *data = reinterpret_cast<float*>(map.data);
projectm_->pcm()->addPCMfloat_2ch(data, samples_per_channel);

Gustavo L Conte

if (projectm_) {
    short samples_per_channel = static_cast<short>(map.size / sizeof(size_t) / 2);
    const short *data = reinterpret_cast<short*>(map.data);
    projectm_->pcm()->addPCM16Data(data, samples_per_channel);
  }

I think I finally got it. Output is finnally as expected, interacting with the song. We only have to divide by sizeof size_t, in samples_per_channel!!!! Now its working just like when I use the pulseaudio Capture code. (also I've removed the resetGL on the render loop)

Screenshot from 2024-07-05 16-52-13.png
Zylot - Crosshair Dimension (Light of the Ages)

Now we need to fix the projection / OpenGL extensions or whatever is wrong that brakes rendering of plugins.