A WSDL Endpoint for Yahoo! REST Api

02 Feb 2008

As I am currently learning .NET and taking interest in SOA, I was searching for free web services using WSDL as their access point.

I headed for Google website.

Google Search Api

I quickly reached Google developer page only to find out that the Search API was deprecated and replaced by the Google AJAX Search API. The search web service is still available, but Google don’t issue key to use it anymore, making it unusable for new users like me. I found many blog posts criticizing this decision, but this is not the point here.

I headed to my second choice search engine: Yahoo!

Yahoo! REST Api

As you might know, Yahoo! provides a REST Api to use their web search service. The call is made using a simple URI with GET parameters, and the response uses a custom XML format.

In my opinion, it is a shame that their don’t offer a WSDL entry point, as it is a W3C open standard. It also forces each developer to write custom XML parsing code in order to process the answer, while there are many Api to call and process responses from WSDL format services without even seeing any XML.

Again, I stumbled upon may blog posts criticizing that decision, but again, it is not the point here. I had to go for my third search engine choice: Live Search (formerly MSN Search).

Live Search

On Windows Live Dev page, I crawled to find information on the search API. I found that the site is mess, as they recently changed name from msn to live. However, I managed to find what I was looking for: a WSDL file.

After a fight with Live account setting page, I finally found a way to obtain a Application ID that is needed to get a response from their web service.

I was now able to start coding my test application, at last!

First Test Application

Inspecting Live Search response format, I quickly wrote up two structs to hold search results.

The first one was designed to hold a result from a search:

public struct SearchResult
{
    public string Title;
    public string Summary;
    public string Url;
    public string DisplayUrl;
    public string Date;
}

The second one was designed to hold a set of SearchResult and an eventual error message:

public struct SearchResponse
{
    public List ResultSet;
    public string Error;
}

I added the WSDL file URI as a web reference to my project, and wrote up a function in order to call the Live Search web service:

public SearchResponse LiveSearch(string query)
{
    SearchResponse rs = new SearchResponse();
    rs.ResultSet = new List();
    SearchResult sr;
    try
    {
        com.msn.search.soap.SearchRequest req = new com.msn.search.soap.SearchRequest();
        com.msn.search.soap.SourceRequest[] srcreq = new com.msn.search.soap.SourceRequest();
        srcreq = new com.msn.search.soap.SourceRequest();
        srcreq.Source = com.msn.search.soap.SourceType.Web;
        req.Requests = srcreq;
        req.AppID = LiveApiID;
        req.Query = query;
        req.CultureInfo = "en-US";
        foreach (com.msn.search.soap.SourceResponse response in new com.msn.search.soap.MSNSearchService().Search(req).Responses)
        {
            foreach (com.msn.search.soap.Result result in response.Results)
            {
                sr = new SearchResult();
                sr.Title = result.Title;
                sr.Summary = result.Summary;
                sr.Url = result.Url;
                sr.DisplayUrl = result.DisplayUrl;
                if (result.DateTime != null) sr.Date = result.DateTime.ToString();
                rs.ResultSet.Add(sr);
            }
        }
    }
    catch (SoapException soapex)
    {
        rs.ResultSet.Clear();
        rs.Error = soapex.Detail.InnerText;
    }
    catch (Exception ex)
    {
        rs.ResultSet.Clear();
        rs.Error = ex.Message;
    }
    return rs;
}

A few exceptions later, my Windows Form test application was working fine. But I wanted more.

Second Test Application

During my search for a Yahoo! search Api, I stumbled upon this article , that gave me an idea: why not usie .NET to build a webservice that would do the search and return results using Yahoo! REST Api?

Looking at wat was returned by Yahoo! REST Api, I slightly modified my initial SearchResult struct:

public struct SearchResult
{
    public string Title;
    public string Summary;
    public string Url;
    public string ClickUrl;
    public string DisplayUrl;
    public string ModificationDate;
    public string MimeType;
    public string Source;
}

I then created a new ASP.NET project to hold my web service, copied the two struct into the new class and wrote a function to call Yahoo! RESP Api, parse it using XPath and fill the responses into a SearchResponse:

[WebMethod]
public SearchResponse YahooSearch(string query, int maxResult)
{
    SearchResponse rs = new SearchResponse();
    rs.ResultSet = new List();
    SearchResult sr;
    try
    {
        HttpWebRequest req = WebRequest.Create("http://api.search.yahoo.com/WebSearchService/V1/webSearch?appid=" + YahooApiID + "&query=" + query + "&results=" + maxResult.ToString()) as HttpWebRequest;
        using (HttpWebResponse res = req.GetResponse() as HttpWebResponse)
        {
            XmlDocument doc = new XmlDocument();
            XmlNamespaceManager ns = new XmlNamespaceManager(doc.NameTable);
            ns.AddNamespace("yns", "urn:yahoo:srch");
            doc.Load(new StreamReader(res.GetResponseStream()));
            foreach (XmlNode node in doc.SelectNodes("/yns:ResultSet/yns:Result", ns))
            {
                sr = new SearchResult();
                sr.Title = node.SelectSingleNode("yns:Title/text()", ns).Value;
                sr.Summary = node.SelectSingleNode("yns:Summary/text()", ns).Value;
                sr.Url = node.SelectSingleNode("yns:Url/text()", ns).Value;
                sr.ClickUrl = node.SelectSingleNode("yns:ClickUrl/text()", ns).Value;
                sr.DisplayUrl = node.SelectSingleNode("yns:DisplayUrl/text()", ns).Value;
                sr.ModificationDate = node.SelectSingleNode("yns:ModificationDate/text()", ns).Value;
                sr.MimeType = node.SelectSingleNode("yns:MimeType/text()", ns).Value;
                rs.ResultSet.Add(sr);
            }
        }
    }
    catch (SoapException soapex)
    {
        rs.ResultSet.Clear();
        rs.Error = soapex.Detail.InnerText;
    }
    catch (Exception ex)
    {
        rs.ResultSet.Clear();
        rs.Error = ex.Message;
    }
    return rs;
}

I am not new to XML, but still, I had trouble with namespaces on this one. I know, I know, shame on me…

Anyway, that was working! I then copied Liave Search function into the ASP.NET class in order to provide access to both services trough WSDL, and wrote up a small Windows Form application to do some testing.

Conclusion

I don’t think there is much to conclude from this experiment. To most experienced .NET developers, this will look like a piece of cake, but as I said this is all new to me. I learned some things along the way, so I think it was worth it.

If you want to let me know what you think, that you found mistakes or you want to enlighten me with your knowledge, please drop a comment.