All times are UTC+02:00




Post new topic  Reply to topic  [ 11 posts ]  Go to page 1 2 Next
Author Message
PostPosted: Mon Jul 04, 2011 1:27 am 

Joined: Fri May 13, 2011 11:21 pm
Posts: 52
There is an interesting function on ST's RDS capability, the one that we can tell ST to read from a specific text file. I though that it would be really nice to include current weather data from a website, like outside temperature or next day forecast.
The data can be updated every, let's say, 5min and be displayed on PS text.

First of all you have to choose the weather info provider. Currently I'm using my city's online meteo station that uploads raw data to a website. Don't worry, you can use Google's secret weather API, so that's what I'm going to explain, below.

I'm getting the raw data from Google (actually it's Weather Underground data) every 5mins using the superb wget (coming from linux, there is a Win version).
To extract temperature from all the data, I'm telling the batch file to search for a specific location inside the raw mess of numbers and get a specific number of characters. Then I tell the batch proggy to output just the temperature to a single text file. At the end, I setup ST to display the text file into the RDS PS.

So, you will need:
1. the wget program, from http://users.ugent.be/~bpuype/wget/
2. the following batch program. Call it whateveryoulike.bat and place it at the same folder with wget.exe Remember to replace YOURCITYGOESHERE to the one you are interested in. You can test it in your browser, you should see an XML file.
Code:
@echo off
del googleout.txt
del googledata.txt
wget -O googledata.txt -F "http://www.google.com/ig/api?weather=YOURCITYGOESHERE&hl=en"
setlocal EnableDelayedExpansion
for %%a in (googledata.txt) do (
for /f "tokens=1 delims=" %%b in (%%a) do (
set rec1=%%b
set ord=!rec1:~482,2!
echo !ord! >> googleout.txt
)
)
As you can see, the process goes like:
Firstly, I delete the files googleout.txt and googledata.txt that may have been created before. So I have no previous weather data at the moment, aka clean start.
I'm running wget so I can get the URL http://www.google.com/ig/api?weather=YO ... HERE&hl=en and I'm outputing the information in the googledata.txt, in English language.
I'm telling the batchfile to search for character location 482 and get the 2 following characters that contain the temperature in Celcius. Then I output just the 2 characters to googleout.txt

Now, at the same directory as the wget.exe and my batch file you should also have googleout.txt and googledata.txt

Go to ST and add at the PS box, the path to the googleout.txt like:
Code:
2s:Temp \F"c:\yourpathhere\googleout.txt"
You are good to go!
Don't forget to make the .bat batch file to be run automatically every X minutes, so you can get fresh data! (Windows Task Scheduler)

Some thoughts:
I know it's not the perfect way to do all that stuff, but hey, it works! 8-)
I wish you could also include weather forecast for the following days. It would be cool to display on the RDS the text like "Tomorrow will be Sunny, Cloudy etc" BUT we don't know how many characters to capture from the googleout.txt file (Any script expert to help on that?)
You have to test in your browser the URL asking Google for the weather data, so you are sure that it get's correctly your city's name.
For geeks: I know that the -F flag on wget is not really needed, but hey! switches and options are cool :p
I've written before that I'm not using Google weather data because they are not accurate for my city. I'm actually using a text file which provides tons of weather info, like humidity, winds, rain etc. Feel free to change the URL in the script and make sure that the batch get the correct characters for the info you want to display.

I hope you find it useful 8-)


Last edited by lef on Mon Jul 04, 2011 1:54 am, edited 1 time in total.

Top
   
PostPosted: Mon Jul 04, 2011 1:41 am 
User avatar

Joined: Tue Mar 17, 2009 2:56 pm
Posts: 4163
Nice idea...
I already done that on RDS in ST year ago ..
Simply using "Weather Watcher". Software can make cuurrent stats into txt file. You can even set what to be in that txt, temperature or "feels like" temperature .. or else ...then simly call into ST-RDS.

I am not using it anymore because they sold themselfs to another company and now my country is not supported anymore...
anyway .. maybe something is changed .. need to check...


Top
   
PostPosted: Mon Jul 04, 2011 1:50 am 

Joined: Fri May 13, 2011 11:21 pm
Posts: 52
I suppose Google's weather API is working correctly for your city?!


Top
   
PostPosted: Mon Jul 04, 2011 11:52 am 
Site Admin
User avatar

Joined: Mon Mar 17, 2008 1:40 am
Posts: 11211
Hm, cool!

I would change 2 things in the script:

1. Write the data to a temporary file, then in the last step overwrite the actual .txt file.
Reason: This way there's (almost) always data available. As it is now, if Stereo Tool reads the txt file when you're just processing the data, you'll get 'Temp:' with nothing.

2. Reading at a specific location in the file is dangerous: Any of the values before it could have a different length (OK, humidity probably won't drop below 10%, but still).
I would suggest that you install the program 'sed'. That's also a Linux utility but there's a Windows version too.
Example:
Code:
echo "<lots><of><xml><stuff><temp>20</temp><more><xml><stuff>" | sed "s/.*<temp>//" | sed "s/<\/temp>.*//"
Output:
Code:
20
Sed uses regular expressions, in short what I'm doing here:
s/a/b/ - substitute a by b.
So I substitute .*<temp> by nothing.
.* stands for:
. = ANY character
* zero or more
This matches EVERYTHING upto <temp>. Then I replace that by nothing.

The 2nd one needs a \ before the / in /temp because otherwise it would be seen as the separation character / :-)

I hope this is a bit clear...


Top
   
PostPosted: Mon Jul 04, 2011 6:17 pm 

Joined: Fri May 13, 2011 11:21 pm
Posts: 52
Nice thoughts, Hans! Yes, the temporary file can be used also.

To be exact, the batch file was originally created to get data from this kind of raw data file:
Code:
12345 6.2 5.8 270 26.6 15 1014.0 0.0 0.0 309.2 0.000 0.000 25.0 34 0.0 0 0.0 0 0 0.0 129.2 103.6 385.2 -100.0 -100.0 -100.0 -100 -100 -100 19 11 47 wmr200-19:11:47 0 0 4 7 100 100 100 100 100 100 100 26.6 23.9 28.0 14.5 0 Dry 0.0 8 8 8 8 8 8 8 8 5 5 5 5 5 5 6 6 6 6 6 6 14.8 -2.2 13497.2 4/7/2011 25.9 12.4 28.0 14.5 0.0 5 6 6 6 10 6 6 9 6 7 27.5 27.4 27.4 27.3 27.1 27.2 27.1 26.8 26.8 26.6 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 26.6 14.5 25.7 11.9 0 --- --- 259 0 0 24.5 -100.0 100 100 100 38 100 0.0 25.0 23.6 23.9 1014.0 1011.0 13 18:39 18:03 27.0 13.7 9.7 -3.0 8 2011 278.3 -1 1 -1 217 217 262 232 244 216 249 253 243 260 0.0 0.0 7.4 13.6 39.63861 -20.86222 0.0 68 14 0.0 06:10 !!C10.37Q!!
So, I'm getting the weather info and variables I need just from the position of the character each time.

I thought I should demonstrate Google's XML API since it can be used by almost anyone, everywhere in the world.

How about adding in ST's RDS section something about the weather? People can write down in a box their city, and you will have to program the background fetching of the weather from that API. I know that's an extra and not common feature that somebody might need, but it would be nice for somebody to have it ready "out of the box". :)


Top
   
PostPosted: Mon Jul 04, 2011 7:42 pm 

Joined: Fri May 13, 2011 11:21 pm
Posts: 52
For Google Weather API
using wget from http://users.ugent.be/~bpuype/cgi-bin/f ... t/wget.exe
and sed from http://sed.sourceforge.net/local/downlo ... indows.zip

Update to Hans's code using sed:
Code:
sed 's/.*temp_c data="\(..\).*/\1/' googledata.txt
Sed is searching for temp_c data=" and then it prints out [s]the 2 following characters[/s] the temparture, [s]which are the numbers only.[/s]
googledata.txt is the XML input file we downloaded

Of course we can make the clean text file containing the temperature, so that ST will use it as:
Code:
sed "s/.*\<temp_c data=\"\([0-9\-]\+\)\"\/>.*/\1/" googledata.txt > googleout.txt
So the complete batch script should be:
Code:
@echo off
del tempout.txt
del googledata.txt
wget -O googledata.txt -F "http://www.google.com/ig/api?weather=YOURCITYGOESHERE&hl=en"
sed "s/.*\<temp_c data=\"\([0-9\-]\+\)\"\/>.*/\1/" googledata.txt > googleout.txt
)
)
I'm still using the delete commands at the start of the script. The new script needs wget and sed to work.
Remember to check once in your browser the URL:
Code:
http://www.google.com/ig/api?weather=YOURCITYGOESHERE&hl=en
updated using Hans's proper way to capture temperature


Last edited by lef on Mon Jul 04, 2011 9:48 pm, edited 4 times in total.

Top
   
PostPosted: Mon Jul 04, 2011 9:32 pm 
Site Admin
User avatar

Joined: Mon Mar 17, 2008 1:40 am
Posts: 11211
Since the temperature data is formatted as:
Code:
<temp_c data="21">
(and the temperature DOES NOT HAVE TO BE 2 CHARACTERS, it can be 1 or even 3 if the temperature is below 0!)

I would go for this one:
Code:
sed "s/.*\<temp_c data=\"\([0-9\-]\+\)\"\/>.*/\1/
And if you - like me - don't like temporary files, this one-liner returns the number for you:
Code:
lynx --dump "http://www.google.com/ig/api?weather=Veghel&hl=en" | sed "s/.*\<temp_c data=\"\([0-9\-]\+\)\"\/>.*/\1/
(I happened to have the text-only browser lynx installed on my system, so I used it instead of wget. But you can use either).

Ideally you should write a batch file that takes the city and wanted field as arguments, and then returns that. I don't know (haven't really looked either) how to find the characters upto the first " - but the following batch file does it for numbers:
Code:
@ECHO OFF
lynx --dump "http://www.google.com/ig/api?weather=%1&hl=en" | sed "s/.*\<%2 data=\"\([0-9\-]\+\)\"\/>.*/\1/
Usage:
googleweather.bat Veghel temp_c

Output:
21


@lef: You're fast... I was still typing!


Top
   
PostPosted: Mon Jul 04, 2011 9:53 pm 

Joined: Fri May 13, 2011 11:21 pm
Posts: 52
Hans, your sed skills are far better than mine. Any way that we can capture the current weather from the XML? I mean only the word between the " " at:
Code:
<current_conditions>
<condition data="Clear"/>


Top
   
PostPosted: Mon Jul 04, 2011 10:57 pm 
Site Admin
User avatar

Joined: Mon Mar 17, 2008 1:40 am
Posts: 11211
Hm, apparently the Windows sed version doesn't support some parts of regular expressions (.*? should match the least possible number of characters, but ? is not supported. And ^" should match anything except for ", but it matches ^ and "...).

So the best I can come up with is this:
Code:
@ECHO OFF
lynx --dump "http://www.google.com/ig/api?weather=%1&hl=en" | sed "s:.*\<%2 data=\"\([0-9\-a-zA-Z\ \t]\+\)\"/>.*:\1:"
Note that this will give you the 1st match, which might not be what you want! (In that case you'll need to extend your sed string to also match a lever higher).


Top
   
PostPosted: Wed Jul 06, 2011 1:31 am 

Joined: Fri May 13, 2011 11:21 pm
Posts: 52
I've switched over to awk and I'm getting the results much easier. ;)


Top
   
Display posts from previous:  Sort by  
Post new topic  Reply to topic  [ 11 posts ]  Go to page 1 2 Next

All times are UTC+02:00


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to:  
cron
Powered by phpBB® Forum Software © phpBB Limited