More than one year after Google discontinued the SOAP Search API, it finally got a proper replacement. The AJAX Search API can now be used from any Web application, not just in JavaScript. The other two Google AJAX APIs for feeds and translations were updated for non-AJAX use, as well.

"For Flash developers, and those developers that have a need to access the AJAX Search API from other Non-Javascript environments, the API exposes a simple RESTful interface. In all cases, the method supported is GET and the response format is a JSON encoded result set with embedded status codes."

"Using the APIs from your Flash or Server Side framework couldn't be simpler. If you know how to make an http request, and how to process a JSON response, you are in business," says Mark Lucovsky. Here's a simple example for web search:
http://ajax.googleapis.com/ajax/services/search/web?v=1.0&q=Earth%20Day

There are some differences between the old SOAP API and the REST one.

PROs:
- the new API doesn't require a key
- there's no limitation for the number of queries
- it's much easier to use
- you can use the REST API for web search, but also for image search, news search, video search, local search, blog search and book search.

CONs:
- you need to send "a valid and accurate http referer header"
- you can only get up to 8 results in a single call and you can't go beyond the first 32 results
- the terms of use are pretty restrictive: for example, you need to attribute the results to Google and you are not allowed to change the order of search results.

It's interesting to notice that Yahoo's search APIs are more developer-friendly and, although they require an application ID and have some usage limitations (5,000 queries per IP per day), they offer more features and they are more flexible, by also including XML output. Another important difference is that Yahoo doesn't require "a valid and accurate http referer header".

Philipp Lenssen suggests that it's much easier to just screenscrape the results, but search engines could change their code or block your requests.

Update. Check this excellent interview with Mark Lucovsky, who mentions that the API has been available for almost two years, but it wasn't officially documented:

Google made a lovely app for Facebook that lets you search the web and share the results with your friends. Your queries are automatically included in Facebook's mini-feed, so your web history can be shared with your friends. There's also a page that showcases popular results found by other Facebook users.

The application has been created using Google's AJAX Search API, the only search API still supported by Google.


At the moment, Google only uses your web history to personalize search results. Maybe in the future you'll be able to share some parts of your logs with your friends (for example, your bookmarks) and obtain better search results by using information from the profiles of your contacts. Yahoo tried to do this with MyWeb 2.0, but failed.

With the release of MyWeb 2.0, Yahoo has added an extensive array of new features focused on community-based searching and sharing of information. "It basically enables people to tap into each other's personal web by searching their trust network of friends," said Eckart Walther, vice president, product management, Yahoo. (...)

Yahoo has also developed a new relevance algorithm called "MyRank" for MyWeb 2.0. "It's a new search engine that we wrote that can search across thousands of nodes and millions of pages in a trust network," said Walther. Unlike PageRank and other link analysis techniques used by general-purpose search engines, MyRank is designed to ferret out clues to relevance based on the pages you and your community have saved to MyWeb 2.0.

Google built a search interface optimized for iPhone that uses the AJAX Search API and it's available at www.google.com/uds/samples/iphone/isearch.html. The interface has all the limitations of the API (for example, you can only see the first 8 results), but it's more usable on a phone.

Mark Lucovsky's team has also added image search to the API, which now includes support for web search, custom search engines, image search, Google News, blog search, book search, Google Maps and video search. Until now, the AJAX Search API was mostly used to build widgets for blogs and personalized homepages and didn't successfully replace the discontinued SOAP API.

Update: A Google spokesperson asked me to clarify that "this is a developer demo, not a new Google service". Google only wanted to show how easily developers can use Google search in their iPhone apps.

If you liked the way Google implemented search in most of their official blogs, now you can get something similar without writing complicated code. Blogger blogs that migrated to layouts can include a search widget that shows results from the blog, its blogroll, the pages linked from the blog and the web. The feature uses the AJAX Search API and Google Custom Search, so the search results are loaded at the top of the current post, without reloading the page.

The widget is available at Blogger in Draft, the place where Blogger showcases experimental features.


The widget may show sponsored links, but you can't earn money from them. In fact, Google doesn't earn money from the ads either and includes them for free. "Ads in the AJAX Search API are basically an experiment still. We are trying to figure out if ads are useful in this model, and how do our customers want to use the ads and benefit from the ads? We don't know the answers yet. Yeah, Google is not benefiting from these ads, because advertisers are not being charged for them. The only guys that actually benefit, to be honest with you here, are the advertisers. They are basically getting free impressions," explained in an interview Mark Lucovsky.

Blogger also added a poll widget and enclosures for podcasts and videocasts. "Enclosure links let you turn your blog into a podcast. If you've uploaded your audio or video to the web, you can link to it as an enclosure so that your readers can subscribe and download it using iTunes or another podcatcher."

Google has an interesting patent that describes the philosophy behind Google AJAX Search API (also known as user-distributed search), a way to bring the search results to the user.

In an increasingly networked world, users frequently use online sources to create and exchange information. Email, instant messaging (IM), message boards, websites, and blogs are all existing communication technologies through which users can create and distribute content to other users. Frequently, in creating such content, a user may wish to reference other online information sources. For example, a user authoring an email may use a browser to navigate to a web page that the user would like to reference in the email, copy the link (e.g., the uniform resource locator (URL)) from the browser to a "clipboard," and then paste the link from the clipboard into the email. In this manner, the user can create an email message that contains links that are accessible by an eventual reader of the email.

Search engines are a popular tool through which users enter a search query describing information of interest and receive back documents or links to documents that relate to the search query. Frequently, when "researching" content for an email message, IM message, message board post, website post, or blog post, the user may perform one or more searches using one or more search engines to locate online documents relevant to the content. The user may then copy a link into the document using the above-described method of copying and pasting a link to the document. This process for annotating user created content can be tedious, difficult to perform for average users, and often results in textual links in the final content that can be difficult to read.

The patent suggests implementing a sidebar that lets you perform searches and easily include the results in the post or message. Another interesting idea is to show implicit search results based on what you type.

Instead of waiting for a user to provide a search query, the UDS may automatically generate search queries based on, for example, entity recognition techniques performed using the content entered by the user. (..) Entity recognition techniques are generally known in the art, and may include, for example, techniques designed to recognize entities such as products, places, organizations, or any other entities that tend to be subjects of searches. The entity recognition techniques can be based on linguistic grammar models or statistical models. In one possible implementation, the entity recognition techniques may be particularly adopted to locate terms that correspond to commercial products or terms that define an address, such as a postal address. In other possible implementations, the entity recognition techniques may be particularly biased to locate terms that are associated with a profile of the user, such as profile explicitly generated by the user (e.g., by the user filling out a questionnaire) or a profile automatically generated for the user, such as a profile based on the user's search history or based on documents created by the user.

This could be the first step towards a rich text editor smart enough to suggest relevant information for what you write and to auto-complete recurrent titles, names or ideas. One of the best existent applications for user-distributed search is Linkify, a bookmarklet that lets you place links in a text box using navigational queries.

{via SEO by the Sea}

How many times did you post something in a blog or in a forum, but felt you need to add a link that explains a concept or a link to the homepage of a product? Laurence Gonsalves from Google used Google AJAX Search to create a bookmarklet that performs a search for the selected text from a form and lets you insert a link to one of the top search results for that query.



The bookmarklet seems to work only in Firefox and Opera. You just have to drag the link to your bookmarks toolbar. (Make sure the bookmarks toolbar is visible in your browser. You can make it visible if you go to View/Toolbars and enable Bookmarks Toolbar in Firefox or Personal Bar in Opera. Alternatively, you can just bookmark the link.)


Then everytime you need to quickly add a link in a textarea, type a text that will become both the query and the anchor text, select it, click on "Linkify" in your toolbar and then click on "Create a link" in the Google AJAX search sidebar.

You can use it when you post a comment to a blog, when you write a blog post or in other situations when you write in a textarea. Unfortunately it doesn't work with Gmail or Google Docs & Spreadsheets.

If you want to monitor the latest news about global warming, Google or your favorite basketball team, there's a newsbar gadget for the Personalized Homepage that shows the headlines at the top of the page and news snippets in the gadget container.



There's also a videobar that shows the top 8 search results from Google Video and YouTube for a list of queries. You could include a list of your favorite artists, TV shows or video series (like Google Techtalks) and the videos are always accessible from your personalized homepage (unless, of course, they are removed by the content providers).


These gadgets, but also Google mini search (that lets you search Google without leaving the page) and Google Map search were implemented using Google AJAX Search API. The API lets you build controls and gadgets centered around search, even if this isn't obvious.

Google tried to repeat the success of the Maps API. "We felt that the Maps API was successful because it was very easy to use, and very easy to explain, where you could show somebody a piece of HTML with some JavaScript, and most anybody could get a map on a webpage very quickly, and have some good eye candy, and some good utility by adding that to their page. What I was interested in is that Google is known for search, and we have search across a wide variety of backends. Could we make search as easy to integrate, and as useful, as we have done with Maps?" explains Mark Lucovsky from Google.

They build different solutions that could be just copy-pasted on a site and also these gadgets for Google Personalized Homepage. And because they use AJAX, you can search and see the search results on the same page that contains the gadgets.

Google AJAX Search is an interesting approach to search: make search results more useful by actually integrating them into a web page.

One way to do that is to add some small gadgets that display search results for queries related to your site. You can now add a book bar, a news bar, a video bar or a map. A news bar shows the latest news about your favorite topics, it occupies a small amount of space, it's always up-to-date and hopefully displays relevant news. The downsides are that the widget increases loading time and the animations might become annoying.

It would be interesting if Google combined these solutions with Related Links, that shows news, videos relevant for a page.

Gnucitizen blog has an interesting post about Google Ajax Search API, a tool that allows you to integrate Google Search into your site and let visitors search Google without leaving your site. The post shows that this API could make life much easier for those who write malware and might facilitate their propagation.

"Web worms can use Google's infrastructure to propagate. If a malicious mind finds a vulnerability in WordPress for example and this vulnerability allows SQL Injection, a worm may be written to crawl blogs in search for this vulnerability and embed itself into everything that is vulnerable. Once a user visits an infected blog the worm starts another cycle.

Another worm might be able to crawl random sites and run generic Cross-site Scripting and SQL Injection checks and send the results to their master who will use them to release more advance worms.

Malicious minds can use Google technology and recently discovered vulnerabilities to create a BotNet that can be used for computational tasks, attacks, information gathering and pretty much everything else that the masters can come up with."

Unlike standard worms, JavaScript worms are not easy to detect and can spread rapidly . The author also thinks that in the future the web will be the new arena for malware, and we may need a web anti-virus that monitors visited web pages.

Related:
Cross-site scripting (Wikipedia)
Cross-site request forgery (Wikipedia)
Samy is my hero (MySpace worm)
More about Google Ajax Search API

This project, known as Ajax Search API, wants to transform search results into a commodity. Imagine a site that lets you create your own playlist of videos from Google Video, or a comment form where you can embed information from search results without leaving the page.

"The Google AJAX Search API is an experimental API that lets you integrate a dynamic Google search module into your web pages so your users can mash up Google search results with other content on your site or add search results clippings to their own content."



The phone list from the video is actually a spreadsheet where you can copy results from Google Local. Imagine how this would look in Google Spreadsheets.

There are three gadgets for Google Personalized Homepage that use this API: video search that lets you create playlists based on search results, map search and the combined search (web, blogs, maps, videos).

I think Google UDS will have an interesting future and will be the foundation for many innovative mashups, like Google Maps API is today.

Google has created Google AJAX Search API, a JavaScript library that allows you to embed many flavors of Google Search into your site: Web Search, Local Search, Video and Blog Search. You can combine the search results from all these sources to obtain a beautiful sidebar like this. What's really nice is that you can customize the search box, the sources and the search is triggered on page load. Users of the site can use the search results to paste them in comments or other user-generated content.

Google AJAX Search API supports Firefox, Safari, and IE 6, it's still in an early phase of development and it's likely that it will include ads in the final release. This is probably the best API ever released by Google and it will certainly be more succesful than Google Maps mashups.

So it seems that Google tries hard to open its content to sites - they launched many APIs lately: GData, AdSense API, and now this very powerful Search API.

{Thank you, Garrett French }