Using bit.ly URL Shortening From ASP.NET (REST/JSON)

As part of my 'learning ASP.NET' series of posts, I thought I would discuss using the bit.ly url shortening API. I will add auto shortening to my SpiniTweet application. SpiniTweet simply takes the last song played on WMFO from Spinitron, and creates a nicely formatted tweetable string. You click, 'Tweet This', and then you don't have to re-type your tracks as you are djing on air to let people know what you are playing. I like to include a link to listen to 'MFO live via the web, in case someone wants to tune in. This is the link I want to shorten.

First off, apparently bit.ly doesn't reuse links, so in my case I really don't need to do this programmatically. But where is the fun in that?!?! Plus, I want to eventually add the ability for any link to be shortened in SpiniTweet, not just the WMFO listener link.

After you sign up for bit.ly, you get your API key in your profile. This is needed because they limit the total number of connects at one time. Next, check out the documentation wiki! bit.ly uses a REST api. If you really want details you can go read that linked wiki page. The bit.ly REST api is simple. We will be most interested in the 'shorten' command, but there are others for doing things like expanding links and getting stats.

Our goal will be to send a url similar to this:

http://api.bit.ly/shorten?version=2.0.1&longUrl=http://cnn.com&login=bitlyapidemo&apiKey=R_0da49e0a9118ff35f52f629d2d71bf07

We can do that with a little chunk of code like so:

    private String GetShortenedURL(String inURL)
    {
        String shortURL = "";

        String queryURL = "http://api.bit.ly/shorten?version=2.0.1&longUrl=" + inURL + "&login=bitlyapidemo&apiKey=R_0da49e0a9118ff35f52f629d2d71bf07";

        HttpWebRequest request = WebRequest.Create(queryURL) as HttpWebRequest;

        using (HttpWebResponse response = request.GetResponse() as HttpWebResponse)
        {
            StreamReader reader = new StreamReader(response.GetResponseStream());

            String jsonResults = reader.ReadToEnd();

            int indexOfBefore = jsonResults.IndexOf("shortUrl\": \"") + 12;
            int indexOfAfter = jsonResults.IndexOf("\"", indexOfBefore);
            shortURL = jsonResults.Substring(indexOfBefore, indexOfAfter - indexOfBefore );
        }

        return shortURL;
    }

Note that the results come back via JSON, but you can also request XML. While it may seem difficult at first, it actually is very simple. I thought initially that I needed to figure out how to parse the JSON output. I found that .NET 3.5 had support for a JSON serialization class. However, in this case, all we care about is the "shortUrl" output. And since it is returned as a string, why not just parse it out ourselves.

Done! Life really is Too Short.

From Windows Forms to Web Forms: Enough The Same To Be Frustrating

In my previous entry I created an application using C# in .NET, using windows forms. While very basic, it performs its task well. Has a 'tweet length' that updates when text is entered, and uses a timer to update the text of the tweet to reflect the latest song being played from your Spinitron playlist. Unfortunately, the radio station's computers use OpenSUSE, so even my target audience (while a small one), can't really use it without bringing in a windows machine.

All the cool kids are always talking about ASP.NET so I figured I'd take a look. I have a background in C/C++. My day job has kept me busy in a very legacy codebase for the past 10+ years. I have broadened my skill set over the years to include things like Cocoa,MFC, and .NET (windows forms). I am no Vostokov but I can hold my own in a .DMP file, and Windbg is DEFINITELY my favorite debugger. Bottom line is, I feel prepared. However, I wouldn't know a postback from a style sheet. But, how hard can it be? :)

I decided to start by just making a web application of SpiniTweet, the radio station twitter application from my previous post. This way people at the station could use it anywhere, and it would also give me a good intro to ASP.NET. One of the books I owned had a chapter on web forms, so I decided to start there.

Turns out, it seemed one of the principals in designing web forms was to make them similar to windows forms. In fact, I was delighted to find out I could reuse my windows form code! So I quickly just banged out a simple web application version.

Launched VS2008, and SpiniTweet4Web was born! Before I touched the code, I went into the designer and quickly made a similar form. It seemed clunky, after using web forms for so long, but I noticed that it was making the html page as I added controls, so I figured it felt that way cause it was following all the rules of how a web page is arranged. I got the basics done, leaving out the countdown timer, and the tweet length for now:



Ok, so its not pretty, but whatever, I am just learning, and I can deal with that later. After getting that arranged, I took a look in the code. It seemed to make sense. Page_Load() is the same as the Form1() constructor. So I just added my initialization code there. Double-clicking the button allowed me to have a _click() handler. Fantastic! It all makes sense!

First problem I ran across is understanding the postback functionality. For example, in Page_Load(), I called 'UpdateFromSpinitron()' which goes and grabs the latest song played on WMFO, and formats a nice string to be tweeted. If I modify the text and click 'Tweet This', you'd think it would just tweet whatever was in the text box, since I query the tweetMsg text box in the button handler here:

System.Net.ServicePointManager.Expect100Continue = false;
HttpWebRequest request = (HttpWebRequest)HttpWebRequest.Create(TwitterJsonUrl);

string post = string.Empty;
using (TextWriter writer = new StringWriter())
{
   String tweetMsgStr = tweetMsg.Text;

   writer.Write("status={0}", HttpUtility.UrlEncode(tweetMsgStr));
   post = writer.ToString();
   Console.WriteLine("Post: {0}", post);
}

Not so! Each time it does a postback, it calls Page_Load() again, which calls 'UpdateFromSpinitron' overwriting whatever I had changed in the textbox. This was the first, of many, similar but different experiences with ASP.NET. Further, this postback reinititalizing the page seems to be a common theme throughout various controls. Seems like a common issue. You get around it by only initializing if it isn't a postback like so:

protected void Page_Load(object sender, EventArgs e)
{
   if (!this.IsPostBack)
   {
         UpdateFromSpinitron();
   }
   AddCharCount();
}

Easy enough. Now, how do I make a twitterish tweet length counter? This is where stuff started to get difficult. Getting the feedback on a per keyup seems to be very difficult on the server side. You can do 'Autopostbacks', which automatically send a postback anytime the control is updated, but only when you de-focus the control is it updated, and thus posted back to the server. Further, I came to the conclusion that you really don't WANT it to be posting back after every keyup. So what else?

Enter AJAX. What the who? Oh yeah, I think I heard about that. Google, right? Talk about behind the curve. This is what living in a legacy codebase for 10 years will do to you.

So I picked up a couple of the biggest books on ASP.NET 3.5 I could find, both had chapters on AJAX. That taught me the basics of how the implementation works. But I wasn't able to really get my length counter working until I found this webpage. It is a fantastic tutorial talking about exactly what I needed. Apparently you can't get away from learning javascript!

So there you have it...my first attempt at ASP.NET. You can see the complete code for my website at codeplex. And even check it out running here: www.pj4533.com/spinitweet

Just a comment, I was hoping that ASP.NET would allow an identical workflow as working with windows forms. It does look like they are pushing in that direction, but they just aren't quite there yet. This has been exciting to learn, and I plan to write many more web apps now! I also plan to add features to SpiniTweet such as a more sexy look (CSS pages), facebook integration, and URL shortening. I'll post up some blog entries when I do so!

Writing a twitter client in C#

I have a radio show at WMFO, at Tufts University. Its a community/college radio station, and completely freeform! For those interested, you can check my radio show website galacticfractures.com, for downloadable shows and whatnot.

At any rate, I wanted to be able to tweet songs as I played them, including a link to our online stream, so people could easily tune in. I did it by hand for a few weeks (on facebook), but it got to be a pain because we already type in songs into a website called Spinitron.com. Effectively I was typing in songs twice, while I was attempting to dj!

I did some digging around and Spinitron publishes a few RSS feeds for each station it supports. One of the feeds is based on the 'current' playlist, so it made it very easy to figure out the most recently entered song. See here:

   private void UpdateFromSpinitron()
   {
       XmlDocument doc = new XmlDocument();

       HttpWebRequest request = WebRequest.Create(SpinitronRSSUrl) as HttpWebRequest;

       String currentPlaying = "";
       using (HttpWebResponse response = request.GetResponse() as HttpWebResponse)
       {
           StreamReader reader = new StreamReader(response.GetResponseStream());
           doc.Load(reader);
           XmlNodeList nodelist = doc.SelectNodes("/rss/channel/item");
           XmlNode currentPlayingNode = nodelist[0].SelectSingleNode("title");

           currentPlaying = currentPlayingNode.InnerText;
           currentPlaying = currentPlaying.Replace(":"," -");
           currentPlaying = currentPlaying.Replace("'", "");
       }
       tweetMsg.Text = "Now Playing: " + currentPlaying + "  (Listen Live @ WMFO.org: http://bit.ly/lOAWA)";
   }

Unfortunately, Spinitron's XML feed is craaaaaaaaap. The "description" node contains ALL the information?!?! Why use XML at all then!? Spread them nodes out, man! Anyway, I was only interested in the artist and title (which are both in the "title" node, ugg).

From there, I just threw it on a 10 second timer to query the XML feed for the latest track. I didn't want it to automatically tweet the song, however, cause I think that could get noisy and be rude to your followers. I instead put a button for tweeting of the song. Maybe you do only every few songs? Whatever...enough to keep people interested.

Lastly, I borrow some code for sending the tweet...but it was real simple. Basically just opening a connection to the twitter url and sending it on! They make it real simple to send tweets, I didn't even have to mess with the twitter API or anything.

I put the project up on codeplex here: http://spinitweet.codeplex.com/

EDIT: It occured to me that I wasn't quite clear on my comments regarding Spinitrons XML feeds. I think I understand why they didn't break it up beyond Title/Description. Obviously they were fitting RSS. However, why even make XML for this? Noone would read a RSS feed of that in Google Reader, for example. It would make more sense to do a RSS feed of a given users shows, and have the description be the nicely formatted complete playlist. Spinitron even says:

"An RSS feed for an individual playlist isn't really in the spirit of RSS
because, unless it happens to be the current playlist, it's not likely to change
like news or blog feeds do. But it may be useful to someone using a script to
grab the playlist data and process it because RSS XML should be easier to parse
than HTML playlist pages."

So if the point is to allow people to parse the playlists, don't worry about RSS, and just give me a better broken down XML file!

MemSpyy

Recently over at If broken it is, fix it you should, Tess Ferrandez posted about a cool memory application that uses output gathered from windbg to show you information graphically. I have written a similar application a few months back, and thought I would scrub it, and share the source. I originally used VisualStudio 2003, but loaded it into 2008, and verified it all builds & runs. I did get a bunch of warnings.

The most interesting thing about this application is that it uses some DLLInjection code from Robert Kuster to run some heap walking code inside the process being viewed. This is the code in question:

MEMORYSTATUSEX statex;
statex.dwLength = sizeof (statex);

GlobalMemoryStatusEx (&statex);

g_UncommittedSize = statex.ullAvailVirtual;

PROCESS_HEAP_ENTRY heapEntry;
memset(&heapEntry,0,sizeof(heapEntry));

while(HeapWalk(GetProcessHeap(), &heapEntry))    
 g_UncommittedSize += heapEntry.cbData;        

Fairly straightforward code, but I can’t figure out how to get that information out of proc.

I thought this would be a good chance to try out CodePlex as well, so I uploaded the source there. CodePlex is pretty nice, basically very similar to GoogleCode and other sites. I was initially excited to try out some VisualStudio integration, but I couldn’t get it to install properly, and got frustrated. I ended up just using TortoiseSVN.

MemSpyy on CodePlex

AirportTraffic

To everyone looking for my old project AirportTraffic, I have decided to make this domain point directly to my new blog. Say Goodnight Software, and any projects I do with the name will just be posted to this blog.

As for AirportTraffic, I havn't done much work on it lately. I still have plans to update it, but my focus has been on PC/Windows applications lately, so I havn't done much OSX development. If you are new to AirportTraffic it is a simple utility for OSX 10.5+ that uses SNMP to communicate to an Aiport Extreme base station & retrieve WLAN bandwidth usage statistics.

You can download the source and hack on it yourself by going to the GoogleCode page.

Fixing Mismatched PDBs to Match!

I was checking out the newish website StackOverflow.com, where you can post up technical questions, and other programmers answer them. Its actually fun, cause you can earn reputation points and 'badges' for answering questions and whatnot.

At any rate, I posted up this question about how to modify the age of a pdb file. I asked because our build process has been broken for a number of years, such that we were generating .PDB file that didn't match the corresponding DLLs. It was an unfortuate side effect of having bad dependencies causing our application to relink when it didn't need to. After each release engineering build target there is a packaging step that gathers the images (dlls and exes) needed for that target. However, the PDBs aren't gathered until the end. Therefore, if an image was relinked between those two times, the newly relinked pdb and the original dll would be off by a few hours.

This could be worked around by using the .symopt +0x40 option in windbg. It forced the pdbs to work properly. However, they would not work in VisualStudio. Also, it meant we couldn't run a symbol server, because with +0x40 it would just grab the first pdb of that name it found.

The great news is that I got an answer on stackoverflow! I was pointed to this site http://undocumented.rawol.com/ which contains the pdfs of the fantastic out of print book called "Undocumented Windows 2000 Secrets: A Programmers Cookbook". The author goes into great detail about the PDB file format, and includes code samples. He even updated the website to include information about version 7.0 (what VisualStudio 2003 uses).

So how do you do it? There are potentially 3 places that contain age & signature information that need to match the corresponding image (dll or exe) file. Two of them are easy to find, because the signature is very recognizable. It is in the header of the file, and easily picked out with a hex editor. I even found this before I was pointed to the book website. In fact, the second reference, near the end of the file, is also a signature&age combination, so it is fairly easy to find as well. However, the last one is TRICKY. Each PDB file is a compound file, and is split into 'streams'. Each stream contains specific information. The 3rd stream contains a reference to the age, but not the signature. Since the age is only a simple 4byte integer value, it is very difficult to pick out on its own! But once I knew where to look, it was easy. I have modified a few releases by hand, but I hope to write a program to automate the process.

Mystery? SOLVED!

Lets say you want to set a break point, but only break into the debugger when you havn't gotten to this point via a given code path. You essentially want to only break when a given function is NOT in the stack trace. How do you do this? In VisualStudio (2003 at least), you don't. The magic of windbg!

I like to use scripts cause I can be very verbose, but you could do this on one line as well. First, just set a breakpoint:

bu `mymodule!myfile.cpp:92` "$$><C:\\windbgscripts\\myscript.txt"

That command sets a breakpoint that will save in a workspace, on the file/line specified. Further, it executes the script specified in the quotes whenever you hit this breakpoint.

Now comes the fun part. Whats in the script?

.if ( @@c++(this->;;m_cRef) != 0)
{  
 .foreach (line {k100})  
 {  
 .if ($SPAT("line","*MyClass::MyFunc*")) {gc}  
 };k   
   
}  
.else {gc}

Look at that madness!!!!! Basically, all I am doing here is checking if the refcount of 'this' is zero. If it is, I just continue with the 'gc' command (gc is 'go from conditional breakpoint'). If it isn't zero, I am interested in it. Then I use the .foreach command. .foreach tokenizes anything in windbg. It can take command or external files as input. In this case, I give it the 'k100' command. This dumps a stacktrace with a depth of 100. Each token is assigned to the variable 'line' and the code is executed. The .if statements use the $SPAT command which check if 2 strings are equal. The cool thing is that you can put wildcards in there. So I just put in some strings from code paths which I don't want to break, and then do a 'gc'. This essentially causes me to only break in the debugger when those strings are NOT in my stacktrace!

WOW!