#spectralpainter

[thegiallo-en]

#Devember 2020 - Day 18 - Spectral Painter - Functions

I’ve replaced the code that was generating the images you saw, with the functions I’ve extracted. Now I keep a float matrix and don’t work converting the initial spectrogram image. Having a single line on, it has no difference.

I’ve noticed that without the image normalization the recomputed spectrogram would look like the original, but with normalization it has black values on the line, so probably they are negative values, I’m not sure there can be negative values though.

The recomputed signal has frequency that seems double of the original from the inverse STFT.

TODOs:

In the inverse STFT I should use the window I used for the STFT.

The way I manage the time shift is wrong.

I’ll have to keep a bigger spectrogram and downscale it for visualization

[thegiallo-en]

#Devember 2020 - Day 13 to 17 - Spectral Painter - Inverse STFT

So, I hope you are not superstitious... :D

I’ve been struggling with FFT. Studying I’ve remembered some stuff and I’ve somehow built an inverse STFT, but I’m not sure it’s correct because of the way I deal with the the time shift. I’ll have to make some more tests. For now I’ve extracted two functions that work on float matrices, but I’ve not yet tested them. I have areas of bad stuff at the start and end of the STFT, because there a part of the window is zero. I’ve tried to mitigate that with Hamming window of adaptive width, but it didn’t work. Also the STFT has a lot of bands.

[thegiallo-en]

#Devember 2020 - Day 12 - Spectral Painter - Hamming Window

I’ve added multiplication by a Hamming window. It’s applied in the STFT from bottom left to bottom right.

[thegiallo-en]

#Devember 2020 - Days 10&11 - Spectral Painter - STFT progress

I’ve made some progress on the STFT and iSTFT implementation. I had some sleepy-coding dumb bugs, even on the debug visualization. I still have to check if the STFT is correct. You can see the iSTFT of the top image below it, and it’s STFT at its right.

[thegiallo-en]

#Devember 2020 - Days 7,8,9 - Spectral Painter - STFT

I’ve been reading this online book about DSP for audio, trying to understand how to perform the inverse STFT correctly and how to perform it back, without loss. I’m not there yet. Sadly the compilation times and the lack of hotreload are slowing the process a lot. Also I have to compile in release mode because in debug it’s not usable. Somehow if the timer function takes more than the period, the whole program clogs up, and stops responding to interrupts.

[thegiallo-en]

#Devember 2020 - Day 6 - Spectral Painter - Eureka

Thanks to Howard Antares, from the JUCE forum, I’ve found the terrible bug that was haunting me.

https://forum.juce.com/t/image-pixels-y-problem/43088/11?u=thegiallo

It was an uninitialized variable due to shadowing. I’m used to have all the warnings possible turned on, but in JUCE I didn’t think about setting them, so the compiler was silent and my mind continued to switch between the meanings of the `h` variable. It was both hue and height, at the same time, depending on the place. So strange.

So now I have this correct hue-saturation plot. Hurray!

I was curious about the pattern that was being created by the uninitialization, and in theory the variable would have the value it had the previous iteration, because of the stack. I tried to initialize it manually to that value and the pattern was still there! I’ve made a shadertoy so that the whole world can benefit from my sleepy coding beautiful discovery.

https://www.shadertoy.com/view/wsGfRy

I’ve now added the flags, but there it comes... no warning. I’ve used goldbolt.org to investigate the problem and discovered that `-flto` disables the warnings about uninitialized use. Also Fahien noticed that the variable pointed by the arrow should be the uninitialized variable and not the assigned one.

https://godbolt.org/z/qnd5xh

P.S.: I’d like to highlight the fact that calling the `setPixel` function results in 7ms to write the image, versus the 0.8ms of the buffer direct access, with `-flto`.

[thegiallo-en]

#Devember 2020 - Day 5 - Spectral Painter - Small Refactor

I’ve done a small refactor of the image test code. I’ve added an update function so that the paint function does only the actual painting. This should make the code more readable by people trying to understand my image problem. I hope I’ll get some answers on this problem. If not, tomorrow I thinks I’ll try to build the program under windows.

[thegiallo-en]

#Devember 2020 - Day 3&4 - Spectral Painter - WTF

So... in these two days I’ve achieved... nothing. I did notice that the HSV image was wrong, no y changes. So I’ve investigated, and confirmed problems on y. I’ve tried many things with no result. As last resort I’ve asked on the JUCE forums. I’m really tempted to go back to my opengl code with hotreload and no portability and raw audio management. https://forum.juce.com/t/image-pixels-y-problem/43088