I finally solved this problem.
I introduced a variable that showed the difference of the number of bytes I sent to the stereo tool process, and the number of samples I received from it.
This difference became larger in time, stereo tool seemed to throw away samples!
So I was sending audio in whatever amount the sound card buffer was needing. This seems to cause trouble in the Stereo Tool process. I always got the audio in chunks of 65536 bytes, and it seemed the rest was discarded. Maybe it's a maximum inputstream buffer size of Java, I don't know.
But I got rid of the issues by buffering up the audio, and sending them in chunks of 4096 bytes.