All times are UTC+02:00




Post new topic  Reply to topic  [ 10 posts ] 
Author Message
PostPosted: Wed Nov 20, 2013 5:14 am 

Joined: Sat Sep 10, 2011 12:42 am
Posts: 123
Location: Australia
I have two ST-SA 7.30 instances for FM MPX running on a rack mount I7 based Win7/64 pc equipped with one Marian PCIe card - driver 1.0.8.0, an SSD and 4GB RAM. First ST instance uses one output channel on the Marian card, the 2nd instance uses a different output channel on the same card. Non-Asio mode. PC on a UPS. Only FM output is enabled in ST.

Been running 24/7 about two months now, and twice (~1 month part) I've had the same 'no audio' issue with just the first instance - while the 2nd instance has never been a problem and it continues on unaffected.

When the problem occurs, audio just stops from the 1st instance - it permanently stops. The GUI of the 1st instance looks ok - I can 'see' it is processing the audio ok, and the GUI is responsive to navigation. The only visible sign in ST there may be an issue is that the FM output buffer stays at 100% full. As expected, the Marian mixer channel allocated to the 1st ST instance shows no audio, and the Marian mixer channel allocated to the 2nd ST instance shows audio (which goes to air ok).

If I try RESTART SOUND CARDS in the 1st instance, all the level metering in the GUI freezes permanently, it does not restore audio, but you can still navigate the GUI menus ok. Trying to close/exit the 1st ST instance then totally freezes ST with a WIn7 'program unresponsive' type message. After forcing it to close, relaunching the 1st instance immediately restores audio output and normal behaviour. As I mentioned, while this is all occurring, the 2nd ST instance does not miss a beat.


Need to track this down. I can update to ST 7.31 if needed, and upgrade the Marian driver to latest version - but the fact the 2nd ST instance continues normally on the Marian card is curious. Any ideas? Thanks.


Top
   
PostPosted: Wed Nov 20, 2013 6:12 am 
User avatar

Joined: Tue Mar 17, 2009 2:56 pm
Posts: 4229
It's very strange that happens on one outuput and not on another..
same soundcard same OS.. all same...

I saw it happens on my PC 4-5 times. For some reason i don't have audio at all. Restarting St solves it. Only common thing is
I also use XP where that never happens.


Top
   
PostPosted: Thu Nov 21, 2013 12:51 am 
Site Admin
User avatar

Joined: Mon Mar 17, 2008 1:40 am
Posts: 11425
Hm, whatever it is should be fixable in the software - since a software restart helps it means that the driver didn't 'die'.

Let's try to go through this step-by-step...

First a question: Is there a difference in the settings of the two ST instances?

The audio stops, but you see that it's still processing - this means that the input is still working, and the whole processing chain as well.

FM output buffer stays at 100% (that should be around 50, normally): I don't get this one. Normally, I put data in the output buffer - and if I cannot put any data in because it's full that's a blocking call. Which means that if no data gets removed from the output buffer, Stereo Tool should hang completely. (As I said, I need to check if this is true - but when using the plugin version with sound card output, the sound card output blocks the progression of the processing). Internally, the sound card driver marks blocks of audio as 'played', and when it does I remove them from the buffer. When the buffer stays full it should block the audio - and putting data into the buffer only happens if it's not full (I also need to check this statement - have some doubts about it). The behavior that you're describing should not be possible regardless of what the driver does - which would point at a bug in Stereo Tool. But the fact that it's always the same instance (well, twice... doesn't really prove anything yet) but more importantly what Bojcha wrote (Bojcha: Did you also have this full buffer behavior? Otherwise what you saw must have been something different) that it doesn't happen on WinXP would point at a driver issue.

RESTART then makes all the meters freeze - I would think that that would be because the sound card cannot be closed (which requires the output buffer to be flushed, but that's not happening). If I recall correctly I just set a flag to have the sound cards reset - if I recall correctly I'm doing that in a different thread. When you try to close ST, it will wait for all the sound card buffers to be flushed, and that doesn't happen, which explains the hangup.


I would definitely upgrade the Marian driver. I'll have to check my code to see if I can somehow explain the things that you're describing.


My current 'hunch' is a driver issue, which could potentially be handled better by Stereo Tool. There seem to be more issues like this (older ESI Juli@ drivers were problematic), I think I need to build in some sort of detection that kills any sound card things that might be hanging and restarts it. At least it should close the sound card if the buffer doesn't get emptied after a while. Problem is that it's nearly impossible to test (I would have to be able to reproduce the issue, and if possible without having to wait a month).


Top
   
PostPosted: Thu Nov 21, 2013 6:05 pm 
Site Admin
User avatar

Joined: Mon Mar 17, 2008 1:40 am
Posts: 11425
I'm looking at the code now. What I see is this: If I'm trying to put audio into the output buffer, but no space becomes available withing 0.5 seconds, I'm dropping the block of audio. Do you by any chance remember if the movement of the rest of the screen was slower than normal (because pushing audio would have blocked the whole chain)?

Ok. The one place where it would make sense that it ST hangs due to a sound card playback error would be in the sound card Close call, which waits (infinitely) for the playback thread to be closed. However - Close is also called when you press the Restart button. In the stand alone version that would mean that the Playback thread would hang - the GUI would still function. That seems to be exactly what you saw. I'm assuming that it's caused by a driver error - but I also think now that I can 'force' similar behavior here to see how ST handles it - and to see if it's possible to force a restart of the sound card I/O.


Top
   
PostPosted: Thu Nov 21, 2013 10:20 pm 
Site Admin
User avatar

Joined: Mon Mar 17, 2008 1:40 am
Posts: 11425
I tried forcing an incorrect detection of whether audio was played, but once that happened no processing occurred anymore. Restarting doesn't change that; trying to close causes a hangup. So, it's close, but it's not exactly the same yet. The cause has to be something different...

The behavior that I'm seeing now does look like what I saw with an older Juli@ driver version (but with that version it happened immediately when audio started to play).

I will look further to try to find the cause!


Top
   
PostPosted: Fri Nov 22, 2013 5:07 pm 
Site Admin
User avatar

Joined: Mon Mar 17, 2008 1:40 am
Posts: 11425
I have done a few things.

First, I've added something in my code that acts as if the driver isn't releasing blocks of audio anymore. This basically caused a hangup (not a GUI hangup but the processing was stilling).

Then I looked where hangups occurred:
1 - When sending output to it
2 - When trying to close the sound card (either by pressing RESTART of by closing ST)

Workarounds:
1 - After 5 seconds of no progress, I'm now automatically restarting all sound card things. Which would still cause a hangup due to 2.
2 - If closing fails for 3 seconds, and no progress is made during that time, I'm 'forgetting' all the data that I have sent to the sound card and just close it. This does mean that all the memory that I have given to the sound card driver does NOT get released - in other words a memory leak. The problem is: As long as the sound card driver has not told me that it finished working with these chunks of memory, if I release them horrible things can happen - a memory leak is probably the best alternative here.

Since this is probably not the exact same thing that you ran into, I cannot promise that this solves anything for you - but it might. At least it detects problems and attempts to restart now. (Only for output so far - I might need to do the same thing for input).

After restarting, a message is displayed in the sound card buffer display to show that something has happened.


There was also a non-standard thing that I did (which I had to do to make the FM output run properly from within Winamp). Since it's non-standard and not needed outside of Winamp, I've removed that now for the other builds.


These changes will be available in the next beta version (732-020).


Top
   
PostPosted: Sat Nov 23, 2013 1:16 am 
Site Admin
User avatar

Joined: Mon Mar 17, 2008 1:40 am
Posts: 11425
I was just looking at the sound card input code and I can do something similar (restart if no audio arrives for 5 seconds) there. I will do that soon (probably tomorrow). ASIO too. These changes should make things more robust to driver or programming errors.

Edit: Now also added and tested it for input. Works fine.

I have also added (nearly identical) code for ASIO, but I have not yet tested it - I need to test it on a pc with ASIO (next Monday). But I don't expect any issues since the code is nearly identical to that of the normal input.


Top
   
PostPosted: Sat Nov 23, 2013 7:28 am 

Joined: Sat Sep 10, 2011 12:42 am
Posts: 123
Location: Australia
Hi Hans - Thanks very much for your quick responses and work. I'll upgrade the Marian driver & adopt next ST beta.

Additional info, not sure if its relevant: The 2 ST instances have a different preset, but the sound device and FM Output section settings are the same (except levels & device channel chosen).

I'm using the Axia multichannel AoIPO driver for input of both instances (separate stations), and the pc is at the FM tx site fed via a licenced Ethernet Radio/STL. There will be odd occasions (<typically < 10-30 s a week) where heavy rain fade or hail on the STL path briefly compromises the Axia Livewire audio delivered to the pc. While I don't know how the AoIP driver behaves feeding audio to ST when that occurs, I'd imagine this is not relevant though, as the two occasions where the first ST instance stops outputting audio where during clear sky conditions, plus the audio input to both ST instances appeared to be ok.

I'm not using ASIO, as currently in ST this forces both the input and output device to ASIO, and one or both (?)my devices (input and output) won't allow sharing of an ASIO device on two separate applications. From memory, I recall the Marian driver *may* be coded to permit that but am unsure. I do not the 'Synchronise to output' feature, as for some reason this just does not 'sync' with the AoIP driver, and ST processes too fast resulting in frequent drops of audio.

I was accessing the pc off site via UltraVNC, but I don't recall the response of the pc or 1st ST instance to be noticeably slower when the issue was present.

Indication of a 'restart' your adding to the buffer display is a great idea. Could I suggest it would be helpful shows the time of day that last occurred (or better still logs/shows last xx restart times), so we can get an idea how frequently that occurs or correlate it to a time we had a report of any 'jump' in transmission. Cheers.

Bojcha - interesting you had something that sounds similar with Win7, but are ok on XP. Key difficulty going forward with Xp is that we can't buy it anymore. Ubuntu/Linux would be a possible great alternative in some situations, but in our case there is (not yet) an Axia LW driver for Linux, I'm not sure if ST supports multiple instances, and Marian does not have Ubuntu/Linux drivers anyway. So we are progressively being forced onto Win 7/8 solutions ...


Top
   
PostPosted: Sat Nov 23, 2013 7:45 am 
Site Admin
User avatar

Joined: Mon Mar 17, 2008 1:40 am
Posts: 11425
New version is posted here: viewtopic.php?f=14&t=4989&p=23084#p23084

About XP: True... Unfortunately. I know that Leif also had issues getting the Omnia 9 stable on Win7 embedded (there were hiccups in the audio), that's why - if it hasn't changed since - it uses XP embedded.

I just wanted to type that you can run multiple versions in Linux when I suddenly started to doubt... It uses names for Jack connections, and if I don't do something to change the names for the 2nd instance it might not be able to connect to it. Marian drivers are indeed an issue: They were working on Linux drivers for the PCIe-cards (Seraph AD2 etc.) but the project is currently on hold, so they don't know when they will be finished.

If the issue is indeed Win7 related then whatever happens must be happening outside of Stereo Tool - either in the driver (most likely) or in Windows itself. By the way, you could use ASIO for one of the two ST instances I think? Not 2 indeed - ASIO itself forbids that (which doesn't mean that tit might not be possible to work around it, of course).


Top
   
PostPosted: Sun Mar 23, 2014 11:41 am 

Joined: Sat Sep 10, 2011 12:42 am
Posts: 123
Location: Australia
Hans - Just to let you know that since running the beta you linked above, we have NOT had this issue reoccur. I stopped it once in the interim to further update the beta. So from late November, its been continuously up and solid. Win7/64 Marian card. I have not updated the Marian driver since original post. thanks. AJ


Top
   
Display posts from previous:  Sort by  
Post new topic  Reply to topic  [ 10 posts ] 

All times are UTC+02:00


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to:  
Powered by phpBB® Forum Software © phpBB Limited