Stereo Tool

*FANFARE*

/me whispers softly: density based compression is the end game

My mouth is closed ... for now... i am fighting with my self...

If with density based you mean RMS based, that's the plan...

BETA074:
- Added 'veil'-based clipping detection.

Basically, I'm dropping a veil - with configurable elasticity - on top of the waveform, and every sample that touches (or is within a configurable margin) the veil is considered to be (possibly) clipped - all others are not.

If BOTH methods indicate that a sample is clipped, I consider it as clipped, otherwise I consider it as not clipped.

The effect on the Metallica track is very limited. Jack is Back however is improved a lot.

Note: This is very fresh development code. Which means:
1. The values are not yet saved
2. There's a 'display veil' checkbox, if you enable it the veil is displayed but the audio is broken. The reason for this is that I'm switching off declipping and abuse the right channel to display the veil data of the left channel. I will have to add some extra display to the declipper window later because I currently have no way to display the veil in the input waveform (because the data should be tilted too, and there's no tilted data display). Oh, and currently the veil works on not yet tilted data.

Winamp DSP plugin: http://www.stereotool.com/download/dsp_ ... 20-074.exe
Stand alone version: http://www.stereotool.com/download/ster ... 20-074.exe
VST version: http://www.stereotool.com/download/vst_ ... 20-074.dll
VST version (No SSE2): http://www.stereotool.com/download/vst_ ... 20-074.dll
Command line version: http://www.stereotool.com/download/ster ... 20-074.exe
Linux command line version: http://www.stereotool.com/download/ster ... ETA620-074 NOT AVAILABLE
Linux GUI version: http://www.stereotool.com/download/ster ... ETA620-074 NOT AVAILABLE

TODO:
- Fix loading changed multiband frequencies
- Add buffer and filter for SCA output (SCA1 ok, SCA2 ok)
- Finish AGC improvement - make mono value configurable (replace checkbox by slider) 1 hour --> NO, not needed - anything else needed? -> NO
- Fix Punch
- Check what to do with new filters (such as bass AGC) - keep them, remove them, change them? --> KEEP
- Save new BASS_AGC setting in VST mode
- Loudness: Annoying cracking sound in bass. Slightly present in 5.00, worse in 6.00, maybe even worse in 6.10. Only when bass is too loud. Much worse than in Final Limiter (at same input level!) - so this clearly indicates a bug. Most likely cause: The filter that was added to remove bass artifacts....... - No, it's the louder bass. But it can be fixed by changing some settings. Default settings updated, and behavior for 'not Very strict' improved. Also Deep bass boost and Very deep bass protection are enabled for latency 512 now.
- Fix crash at program close
- Fix VST plugin version (does not run)
- Dynamically drop 'Allow louder highs, even if it causes vibrations' to 0 when bass filter suspects noticeable voice vibrations.
- Reduce Loudness CPU load
- Check and remove static variables
- Finish new de-essing filter (check what to do with the settings, remove at least some!)
- Convert Multiband input to MONO, then use arrays [2][4096] --> should give speedup. - FAILED
- Natural Dynamics: Fix or remove transient boost
- Natural Dynamics: Add expected + strength slider per band
- Finish declipping filter (clipping level detection + level reduction in dB). 1. Figure out why removal of unwanted frequencies causes flat lines at high quality setting with small overlap. This causes distortion, with this fixed repairs are MUCH better. 2. Fix MP3 correction, automatically scale down when this deteriorates the sound. --> TOP part fails!
- Declipping filter: Fix low latency behavior
- Always oversample clipping (configurable)
- Declipping filter: Change detection at lower input levels.
- "Test Right Channel" in "FM Transmitter Calibration" does not work since v6.10 (standalone) (bojcha)
- Declipper window close function - check! - Seems ok
- Scopes black background?
- Add AGC start level
- Declipping filter: Add comparting of sample history to make sure loud bursts are still detected properly (now, with 16 blocks of history, 65 samples are removed even when ignoring just 0.1%). I should also check the current block (probably with a margin of a factor 2).
- Declipping filter: Use reliability of tilt detection to determine movement speed; slowly move to default tilt (configurable!)
- Declipping filter: Far too many samples are marked as 'maybe/probably' clipped. Histogram not used or not cleared?
- Declipping filter: Optimize tilt detection for performance. SSE2 for maximum. And keep separate smaller histograms to determine the maximum (should perform much better).
- Declipping filter: Cleaned histogram must be mirrorred. After cleanup, mirror center is place where SIGMA above == SIGMA below! Only use on the lower end. Then run cleanup again! The result should contain approximately the same amount of data above and below. If not, that indicates incorrect detection.
- Declipping filter: Add a long term histogram, and use maximum of short term and long term start clipping level detection.
- Declipping filter: Check if 'No declipping when tilt cannot be determined' can safely be turned on.
- Declipping filter: Add setting and saving of new sliders (history size, percentage of highest samples to drop, tilt detection range start, end and precision)
- Declipping filter: Check why higher Quality settings causes cracking sounds. Answer: Rounding error, and too low filtering values to remove it.
- Move pre-emphasis to end of processing
- AGC: Separate 'Remove remaining peaks' settings for band 1 and 2
- Channels L/R swap in stand alone version when changing filtering/quality (eldoradofm)
- Declipping filter: Optimize peak matching for performance: Move determination to extra preprocessing step DONE; try to change if statements to min/max or something IMPOSSIBLE.
- Fix changing value of 'final limit' in Multiband when changing Loudness settings.
- Save bandpass filter order
- Save all new settings, also through VST interface
- Declipping filter: Change default values for Maximum distortion and Step size (see Bojcha's post)
- Finish blind interface WAITING FOR FEEDBACK TODO: TEST: Only display this window when needed.
- RDS issue reported here: viewtopic.php?f=15&t=3703&p=11524#p11524 partially solved
- Declipper: Fix loud bass with soft higher frequencies issue. Do something with loudest frequency vs. peak level - if it cannot fit (Madonna, Metallica) there must be clipping, otherwise there might not be. On top of this I should somehow try to preserve deformed bass sounds (if they are not hard clipped). Maybe this will work: Find loudest frequency, if it's very low. Than assume that it's clipped at the maximum restoration level (?); generate a new tone (using lowpass filter) of the same frequency and clip that at some margin below the maximum peak level. Then filter the input signal based on the FFT of the clipped signal. Finally analyze this new signal; if it is indeed clipped the filtered output will resemble a clipped waveform (visible in sample histogram; no tilt issues!) instead of a sine wave. Use the output of this as input to determine how strict the clipping detection for the input waveform should be (eg. based on normal or sinusoid signals). This is all way too difficult and it just won't work. Here's the real solution: Detect if a loud bass sound is present, if so make sure that symmetrical clipping detection is used, possibly with automatic 'maybe clipped' level adjustment. This should only slightly affect Metallica (which still sounds very reasonable with symmetrical enabled), but has a big effect on DBN - Jack is Back (which is far less affected with symmetrical enabled; auto likely/maybe adjustment based on symmetrical should help even more). Using veil solution now, waiting for feedback...
- Longer history for maximum level
- Try to improve reconstruction match (should remove artifacts for low bass + soft highs)
- Fix veil and detilting display (either in main window or in declipper window)
- Save veil settings (STS + VST)
- Update presets? (BASS_AGC etc.) 1 day
- Natural Dynamics: Attempt to reduce number of bands by 2 (memory + cpu DIV 2), using bands with sloppy slopes (no interaction between bands -> memory usage DIV 2, out-of-cache reads gone, hopefull final filter gone too -> CPU DIV 2 again).
- Natural Dynamics: Voices, especially in chorus, still sound weird.
- Natural Dynamics: Smooth out different bands more to reduce low-bitrate-MP3-like sounds.
- Natural Dynamics: Optimize situation with no interaction between bands for performance.
- Natural Dynamics: Save settings in VST version[/color]
- Change version number 1 hour
- Release 1 hour
- Add lowpass filter for stereo signal (will cause a lot of extra latency!) - it might be possible to avoid this latency using a Hilbert transform
- Add smarter clipping detection. Maybe something much simpler suffices: Current clipping detection with threshold + flat line detection
- Declipping filter: Automatically override the 3 clipping level sliders if the clipping level is detected very clearly (clear thin spike in sample value histogram). - NONSENSE, this is already done by the histogram function. But it can be made a BIT better - I think - by automatically LOWERING the 'always clipped' slider if a lot of data is present at the highest few bins (but care is needed for DBN - Jack is Back like tracks)
- Declipping filter: Create a new histogram after declipping, clean up and subtract from original.
- Change stand alone version to 32 bits.
- Optimization: Reduce CPU usage when input is silence.

I found 1 track which still has some issues (from the freakday, dunno the name, it's the one that failed completely in the Omnia 9 flac file). It's WAY better than it was in that version, but it's still worse than without declipping. However given the vast improvement of almost every other track I've tested I think this is acceptable - at least for now.

Track12? David Guetta feat. Akon - Sexy bitch ?
Yes, distorted highs. In Omnia.9 (new clipper) demo sounds bad too. worse then offical demo. I don't know is it used your De-Clipper or no protection on.

btw..
- deClipper works notecable better (except mentioned track) with soft highs on loud lows now. [tested st default settings]
- Natural Dynamic is broken when "after AGC" is ON

Leif is probably still using the declipper that I gave him for the NAB... That's without ANY protection built in (everything above a certain level below the peak level is considered to be clipped). And I know that with my declipper he has set that 'certain level' a lot lower than with the original declipper.

I'll check if the 'veil' margin (currently everything within 10% below the veil is considered clipped) setting can be set more conservative with no effect on the reconstruction - that might help.

I'm going to disable (temporarily remove) Natural Dynamics before releasing version 6.20 - it's definitely not good enough yet to release.

Aha !

So, what do you do except loading the presets ?
I think that this topic is the wrong place (we are off-topic):

viewtopic.php?p=12236#p12236

I've added a filter that, at the end of the reconstruction, for blocks of > 100 clipped samples, checks if the reconstructed signal is close to the original signal (average difference small, total movement between adjacent samples smaller than in original - based on the observation that in such large blocks if reconstruction fails, the result is flatter than the original). If that's the case it takes the original samples instead of the reconstructed ones.

This seems to work well for this particular track (David Guetta feat. Akon - Sexy bitch), and doesn't have any effect on all the other tracks that I tested (well, it does on DBN - Jack is Back, but that's good!).

Unfortunately in the latter track there are no large blocks which are not properly reconstructed that are marked as clipped, so it doesn't improve the sound there - it only affects the bass that was already reconstructed properly.

Building.

BETA075:
- Switching off declipping for large blocks where declipped signal is very close to original (so apparently the signal was not clipped). And only if the reconstructed signal is flatter than the original (that might be a bit too conservative though).

Winamp DSP plugin: http://www.stereotool.com/download/dsp_ ... 20-075.exe
Stand alone version: http://www.stereotool.com/download/ster ... 20-075.exe
VST version: http://www.stereotool.com/download/vst_ ... 20-075.dll
VST version (No SSE2): http://www.stereotool.com/download/vst_ ... 20-075.dll
Command line version: http://www.stereotool.com/download/ster ... 20-075.exe
Linux command line version: http://www.stereotool.com/download/ster ... ETA620-075 NOT AVAILABLE
Linux GUI version: http://www.stereotool.com/download/ster ... ETA620-075 NOT AVAILABLE

TODO:
- Fix loading changed multiband frequencies
- Add buffer and filter for SCA output (SCA1 ok, SCA2 ok)
- Finish AGC improvement - make mono value configurable (replace checkbox by slider) 1 hour --> NO, not needed - anything else needed? -> NO
- Fix Punch
- Check what to do with new filters (such as bass AGC) - keep them, remove them, change them? --> KEEP
- Save new BASS_AGC setting in VST mode
- Loudness: Annoying cracking sound in bass. Slightly present in 5.00, worse in 6.00, maybe even worse in 6.10. Only when bass is too loud. Much worse than in Final Limiter (at same input level!) - so this clearly indicates a bug. Most likely cause: The filter that was added to remove bass artifacts....... - No, it's the louder bass. But it can be fixed by changing some settings. Default settings updated, and behavior for 'not Very strict' improved. Also Deep bass boost and Very deep bass protection are enabled for latency 512 now.
- Fix crash at program close
- Fix VST plugin version (does not run)
- Dynamically drop 'Allow louder highs, even if it causes vibrations' to 0 when bass filter suspects noticeable voice vibrations.
- Reduce Loudness CPU load
- Check and remove static variables
- Finish new de-essing filter (check what to do with the settings, remove at least some!)
- Convert Multiband input to MONO, then use arrays [2][4096] --> should give speedup. - FAILED
- Natural Dynamics: Fix or remove transient boost
- Natural Dynamics: Add expected + strength slider per band
- Finish declipping filter (clipping level detection + level reduction in dB). 1. Figure out why removal of unwanted frequencies causes flat lines at high quality setting with small overlap. This causes distortion, with this fixed repairs are MUCH better. 2. Fix MP3 correction, automatically scale down when this deteriorates the sound. --> TOP part fails!
- Declipping filter: Fix low latency behavior
- Always oversample clipping (configurable)
- Declipping filter: Change detection at lower input levels.
- "Test Right Channel" in "FM Transmitter Calibration" does not work since v6.10 (standalone) (bojcha)
- Declipper window close function - check! - Seems ok
- Scopes black background?
- Add AGC start level
- Declipping filter: Add comparting of sample history to make sure loud bursts are still detected properly (now, with 16 blocks of history, 65 samples are removed even when ignoring just 0.1%). I should also check the current block (probably with a margin of a factor 2).
- Declipping filter: Use reliability of tilt detection to determine movement speed; slowly move to default tilt (configurable!)
- Declipping filter: Far too many samples are marked as 'maybe/probably' clipped. Histogram not used or not cleared?
- Declipping filter: Optimize tilt detection for performance. SSE2 for maximum. And keep separate smaller histograms to determine the maximum (should perform much better).
- Declipping filter: Cleaned histogram must be mirrorred. After cleanup, mirror center is place where SIGMA above == SIGMA below! Only use on the lower end. Then run cleanup again! The result should contain approximately the same amount of data above and below. If not, that indicates incorrect detection.
- Declipping filter: Add a long term histogram, and use maximum of short term and long term start clipping level detection.
- Declipping filter: Check if 'No declipping when tilt cannot be determined' can safely be turned on.
- Declipping filter: Add setting and saving of new sliders (history size, percentage of highest samples to drop, tilt detection range start, end and precision)
- Declipping filter: Check why higher Quality settings causes cracking sounds. Answer: Rounding error, and too low filtering values to remove it.
- Move pre-emphasis to end of processing
- AGC: Separate 'Remove remaining peaks' settings for band 1 and 2
- Channels L/R swap in stand alone version when changing filtering/quality (eldoradofm)
- Declipping filter: Optimize peak matching for performance: Move determination to extra preprocessing step DONE; try to change if statements to min/max or something IMPOSSIBLE.
- Fix changing value of 'final limit' in Multiband when changing Loudness settings.
- Save bandpass filter order
- Save all new settings, also through VST interface
- Declipping filter: Change default values for Maximum distortion and Step size (see Bojcha's post)
- Try to improve reconstruction match (should remove artifacts for low bass + soft highs) ■ FAILED
- Declipper: Fix loud bass with soft higher frequencies issue. Do something with loudest frequency vs. peak level - if it cannot fit (Madonna, Metallica) there must be clipping, otherwise there might not be. On top of this I should somehow try to preserve deformed bass sounds (if they are not hard clipped). Maybe this will work: Find loudest frequency, if it's very low. Than assume that it's clipped at the maximum restoration level (?); generate a new tone (using lowpass filter) of the same frequency and clip that at some margin below the maximum peak level. Then filter the input signal based on the FFT of the clipped signal. Finally analyze this new signal; if it is indeed clipped the filtered output will resemble a clipped waveform (visible in sample histogram; no tilt issues!) instead of a sine wave. Use the output of this as input to determine how strict the clipping detection for the input waveform should be (eg. based on normal or sinusoid signals). This is all way too difficult and it just won't work. Here's the real solution: Detect if a loud bass sound is present, if so make sure that symmetrical clipping detection is used, possibly with automatic 'maybe clipped' level adjustment. This should only slightly affect Metallica (which still sounds very reasonable with symmetrical enabled), but has a big effect on DBN - Jack is Back (which is far less affected with symmetrical enabled; auto likely/maybe adjustment based on symmetrical should help even more). Using veil solution now
- Move veil processing to detilted signal
- Finish blind interface WAITING FOR FEEDBACK TODO: TEST: Only display this window when needed.
- RDS issue reported here: viewtopic.php?f=15&t=3703&p=11524#p11524 partially solved
- Save veil settings and 'not reconstruct if like original' (STS + VST)
- Fix veil and detilting display (either in main window or in declipper window)
- Switch to symmetrical detection if very loud very low frequencies are present
- Longer history for maximum level?
- Update presets? (BASS_AGC etc.) 1 day
- Remove Natural Dynamics
- Change version number 1 hour
- Release 1 hour
- Add lowpass filter for stereo signal (will cause a lot of extra latency!) - it might be possible to avoid this latency using a Hilbert transform
- Add smarter clipping detection. Maybe something much simpler suffices: Current clipping detection with threshold + flat line detection
- Declipping filter: Automatically override the 3 clipping level sliders if the clipping level is detected very clearly (clear thin spike in sample value histogram). - NONSENSE, this is already done by the histogram function. But it can be made a BIT better - I think - by automatically LOWERING the 'always clipped' slider if a lot of data is present at the highest few bins (but care is needed for DBN - Jack is Back like tracks)
- Declipping filter: Create a new histogram after declipping, clean up and subtract from original.
- Change stand alone version to 32 bits.
- Optimization: Reduce CPU usage when input is silence.
- Natural Dynamics: Attempt to reduce number of bands by 2 (memory + cpu DIV 2), using bands with sloppy slopes (no interaction between bands -> memory usage DIV 2, out-of-cache reads gone, hopefull final filter gone too -> CPU DIV 2 again).
- Natural Dynamics: Voices, especially in chorus, still sound weird.
- Natural Dynamics: Smooth out different bands more to reduce low-bitrate-MP3-like sounds.
- Natural Dynamics: Optimize situation with no interaction between bands for performance.
- Natural Dynamics: Save settings in VST version