Build a semantic Search Engine with BOSS

Published
by
Ted
on May 3, 2009
in BOSS Functionality, Relevancy and code samples
. 2 Comments Tags: , , , , , , .

What’s the quickest way to build a semantic search engine? Use BOSS and and the Yahoo! Searchmonkey query filter. You’ll be surprised how much information is available to begin working with.

The Search Monkey Query Filter

This pattern works in the basic Yahoo! search box: searchmonkeyid:com.yahoo.page.uf.{microformat}. It tells Yahoo! Search to only return web pages that include a certain microformat. We can also use this in our BOSS API requests. Here’s a list of structured data surfaced in BOSS via SearchMonkey:

  • hAtom -
    Represents a subset of the Atom syndication format
  • hCalendar -
    Represents calendar dates and events, using a representation of
    the iCalendar
    standard
  • hCard -
    Represents people, companies, organizations, and places, using a
    representation of the vCard standard
  • hReview -
    Represents reviews of products, services, businesses, and
    events
  • XFN – Represents
    human relationships using hyperlinks
  • Geo -
    Represents geograhic coordinates
  • rel-tag – Marks
    up the destination of a hyperlink as an author-designated
    tag
  • adr -
    Represents address information

  • Dublin
    Core
    - Allows developers to specify document metadata
  • FOAF
    - Friend of a Friend specifies personal profiles and social
    networks
  • SIOC
    - Specifies elements in blogs, forums and Q&A sites
  • Other
    supported vocabularies     – See the SearchMonkey
    documentation for a list of other supported vocabularies

Universal BOSS API Query Operators: Apply to Web, Images and News

SearchMonkey in Your BOSS

Now we have the ability to limit our search results to sites that use a particular microformat. We can now ask for this structured data to build our robust search engine We’ll use the SearchMonkey view attributes. There are two variations that define the format of the data.

  • view=searchmonkey_feed (dataRSS format)
  • view=searchmonkey_rdf (rdf format)

The following example shows how each look.

Let’s build a semantic music search engine. We’ll start by filtering for hreview microformats to get music reviews about CocoRosie:  http://boss.yahooapis.com/ysearch/web/v1/cocorosie+searchmonkeyid:com.yahoo.page.uf.hreview?appid=Your_App_Id&format=xml&start=0&count=15&view=keyterms%2Csearchmonkey_feed%2Csearchmonkey_rdf.

We can use the following sample result to begin inserting reviews as well as other structured data. You’ve got the basic foundation for building an application that presents data-rich information to your users. Explore your favorite rdf and micformat queries to come up with new ideas.

Please leave a comment if you are already using this technique in your site or have similar ideas to share with others.

Sample Result

<?xml version="1.0" encoding="ISO-8859-1"?>
<result>
<abstract>
<![CDATA[Contributed by: ant-likekarma on July 13th, 2007 at 6:02 am.
Tekno Love Song by CocoRosie... love song tekno cocorosie.
Other Info. Lifetime rating of 4.83 ...]]>
</abstract>
<clickurl>http://lrd.yahooapis.com/.../**http%3A//www.blogskins.com/info/150260</clickurl>
<date>2009/04/22</date>
<dispurl><![CDATA[www.blogskins.com/info/150260]]></dispurl>
<keyterms>
<terms>
	<term>skin</term>
	<term>Tekno</term>
	<term>Love Song</term>
	<term>download</term>
	<term>BlogSkins</term>
	<term>Skin Info</term>
	<term>fade</term>
	<term>Skin type</term>
	<term>bg</term>
	<term>follow this link</term>
	<term>cocorosie</term>
	<term>Blogger</term>
	<term>tear</term>
	<term>keep well</term>
	<term>Lifetime</term>
	<term>logged-in</term>
	<term>BM</term>
	<term>Random Skin</term>
	<term>Myles</term>
	<term>hmm</term>
</terms>
</keyterms>
<searchmonkey_feed>
<feed>
	<adjunct id="com.yahoo.page.uf.hcard" updated="2009-04-27T16:03:12Z" version="1.1">
		<item rel="dc:subject rel:Card" resource="/me/screamingtragic">
		<type typeof="vcard:VCard" resource="/me/screamingtragic">
			<meta property="vcard:fn" data_quality="255">screaming!tragic</meta>
			<item rel="vcard:url" resource="/me/screamingtragic"/>
		</type>
		</item>
		<item rel="dc:subject rel:Card" resource="/me/animate_">
		<type typeof="vcard:VCard" resource="/me/animate_">
			<meta property="vcard:fn" data_quality="255">animate_</meta>
			<item rel="vcard:url" resource="/me/animate_"/>
		</type>
		</item>
		<item rel="dc:subject rel:Card" resource="/me/madeline_tn">
		<type typeof="vcard:VCard" resource="/me/madeline_tn">
			<meta property="vcard:fn" data_quality="255">madeline_tn</meta>
			<item rel="vcard:url" resource="/me/madeline_tn"/>
		</type>
		</item>
	</adjunct>
	<adjunct id="com.yahoo.page.uf.hreview" updated="2009-04-27T16:03:12Z" version="1.1">
		<item rel="dc:subject rel:Review" resource="http://www.blogskins.com/info/150260">
		<type typeof="review:Review" resource="http://www.blogskins.com/info/150260">
			<meta property="review:title">Tekno Love Song</meta>
			<meta property="review:text">either continue th patern or don't fade the
			background, and all will look better. but it's still lovely!</meta>
			<meta datatype="xsd:dateTime" property="dc:date" data_quality="150">20080421</meta>
			<item rel="review:reviewer" resource="/me/screamingtragic">
			<type typeof="vcard:VCard" resource="/me/screamingtragic">
			<meta property="vcard:fn" data_quality="255">screaming!tragic</meta>
			<item rel="vcard:url" resource="/me/screamingtragic"/>
			</type>
			</item>
			<item rel="rdfs:seeAlso" resource="http://www.blogskins.com/info/150260"/>
		</type>
		</item>
		<item rel="dc:subject rel:Review" resource="http://www.blogskins.com/info/150260">
		<type typeof="review:Review" resource="http://www.blogskins.com/info/150260">
			<meta property="review:title">Tekno Love Song</meta>
			<meta property="review:text">you need stars! I think the bg shouldn't fade off.
			Other than that, it's fabulous</meta>
			<meta datatype="xsd:dateTime" property="dc:date" data_quality="150">20070903</meta>
			<item rel="review:reviewer" resource="/me/animate_">
			<type typeof="vcard:VCard" resource="/me/animate_">
				<meta property="vcard:fn" data_quality="255">animate_</meta>
				<item rel="vcard:url" resource="/me/animate_"/>
			</type>
			</item>
			<item rel="rdfs:seeAlso" resource="http://www.blogskins.com/info/150260"/>
		</type>
		</item>
		<item rel="dc:subject rel:Review" resource="http://www.blogskins.com/info/150260">
		<type typeof="review:Review" resource="http://www.blogskins.com/info/150260">
			<meta property="review:title">Tekno Love Song</meta>
			<meta property="review:text">hmm.. the bg shouldn't be white. should continue
			the pattern thingy... plus, the picture somehow doesn't appeal to me..</meta>
			<meta datatype="xsd:dateTime" property="dc:date" data_quality="150">20070713</meta>
			<item rel="review:reviewer" resource="/me/madeline_tn">
				<type typeof="vcard:VCard" resource="/me/madeline_tn">
					<meta property="vcard:fn" data_quality="255">madeline_tn</meta>
					<item rel="vcard:url" resource="/me/madeline_tn"/>
				</type>
			</item>
			<item rel="rdfs:seeAlso" resource="http://www.blogskins.com/info/150260"/>
		</type>
		</item>
	</adjunct>
	<adjunct id="com.yahoo.page.uf.tag" updated="2009-04-27T16:03:12Z" version="1.1">
		<meta property="tagspace:tag">love</meta>
		<meta property="tagspace:tag">song</meta>
		<meta property="tagspace:tag">tekno</meta>
		<meta property="tagspace:tag">cocorosie</meta>
		<item rel="stag:tagged">
			<type typeof="stag:Tag">
				<meta property="stag:label">love</meta>
			</type>
		</item>
		<item rel="stag:tagged">
			<type typeof="stag:Tag">
				<meta property="stag:label">song</meta>
			</type>
		</item>
		<item rel="stag:tagged">
			<type typeof="stag:Tag">
				<meta property="stag:label">tekno</meta>
			</type>
		</item>
		<item rel="stag:tagged">
			<type typeof="stag:Tag">
				<meta property="stag:label">cocorosie</meta>
			</type>
		</item>
	</adjunct>
</feed>
</searchmonkey_feed>
<searchmonkey_rdf>
<rdf:RDF xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#"
xmlns:h="http://www.w3.org/1999/xhtml">
	<rdf:description rdf:about="http://www.blogskins.com/info/150260">
		<dc:subject xmlns:dc="http://purl.org/dc/terms/"
		rdf:resource="http://www.blogskins.com/me/screamingtragic"/>
	</rdf:description>
	<rdf:description rdf:about="http://www.blogskins.com/info/150260">
		<rel:card xmlns:rel="http://search.yahoo.com/searchmonkey-relation/"
		rdf:resource="http://www.blogskins.com/me/screamingtragic"/>
	</rdf:description>
	<rdf:description rdf:about="http://www.blogskins.com/me/screamingtragic">
		<rdf:type rdf:resource="http://www.w3.org/2006/vcard/ns#VCard"/>
	</rdf:description>
	<rdf:description rdf:about="http://www.blogskins.com/me/screamingtragic">
		<vcard:fn xmlns:vcard="http://www.w3.org/2006/vcard/ns#">screaming!tragic</vcard:fn>
	</rdf:description>
	<rdf:description rdf:about="http://www.blogskins.com/me/screamingtragic">
		<vcard:url xmlns:vcard="http://www.w3.org/2006/vcard/ns#"
		rdf:resource="http://www.blogskins.com/me/screamingtragic"/>
	</rdf:description>
	<rdf:description rdf:about="http://www.blogskins.com/info/150260">
		<dc:subject xmlns:dc="http://purl.org/dc/terms/"
		rdf:resource="http://www.blogskins.com/me/animate_"/>
	</rdf:description>
	<rdf:description rdf:about="http://www.blogskins.com/info/150260">
		<rel:card xmlns:rel="http://search.yahoo.com/searchmonkey-relation/"
		rdf:resource="http://www.blogskins.com/me/animate_"/>
	</rdf:description>
	<rdf:description rdf:about="http://www.blogskins.com/me/animate_">
		<rdf:type rdf:resource="http://www.w3.org/2006/vcard/ns#VCard"/>
	</rdf:description>
	<rdf:description rdf:about="http://www.blogskins.com/me/animate_">
		<vcard:fn xmlns:vcard="http://www.w3.org/2006/vcard/ns#">animate_</vcard:fn>
	</rdf:description>
	<rdf:description rdf:about="http://www.blogskins.com/me/animate_">
		<vcard:url xmlns:vcard="http://www.w3.org/2006/vcard/ns#"
		rdf:resource="http://www.blogskins.com/me/animate_"/>
	</rdf:description>
	<rdf:description rdf:about="http://www.blogskins.com/info/150260">
		<dc:subject xmlns:dc="http://purl.org/dc/terms/"
		rdf:resource="http://www.blogskins.com/me/madeline_tn"/>
	</rdf:description>
	<rdf:description rdf:about="http://www.blogskins.com/info/150260">
		<rel:card xmlns:rel="http://search.yahoo.com/searchmonkey-relation/"
		rdf:resource="http://www.blogskins.com/me/madeline_tn"/>
	</rdf:description>
	<rdf:description rdf:about="http://www.blogskins.com/me/madeline_tn">
		<rdf:type rdf:resource="http://www.w3.org/2006/vcard/ns#VCard"/>
	</rdf:description>
	<rdf:description rdf:about="http://www.blogskins.com/me/madeline_tn">
		<vcard:fn xmlns:vcard="http://www.w3.org/2006/vcard/ns#">madeline_tn</vcard:fn>
	</rdf:description>
	<rdf:description rdf:about="http://www.blogskins.com/me/madeline_tn">
		<vcard:url xmlns:vcard="http://www.w3.org/2006/vcard/ns#"
		rdf:resource="http://www.blogskins.com/me/madeline_tn"/>
	</rdf:description>
	<rdf:description rdf:about="http://www.blogskins.com/info/150260">
		<dc:subject xmlns:dc="http://purl.org/dc/terms/"
		rdf:resource="http://www.blogskins.com/info/150260"/>
	</rdf:description>
	<rdf:description rdf:about="http://www.blogskins.com/info/150260">
		<rel:review xmlns:rel="http://search.yahoo.com/searchmonkey-relation/"
		rdf:resource="http://www.blogskins.com/info/150260"/>
	</rdf:description>
	<rdf:description rdf:about="http://www.blogskins.com/info/150260">
		<rdf:type rdf:resource="http://purl.org/stuff/rev#Review"/>
	</rdf:description>
	<rdf:description rdf:about="http://www.blogskins.com/info/150260">
		<review:title xmlns:review="http://purl.org/stuff/rev#">Tekno Love Song</review:title>
	</rdf:description>
	<rdf:description rdf:about="http://www.blogskins.com/info/150260">
		<review:text xmlns:review="http://purl.org/stuff/rev#">either continue th patern or
		don't fade the background, and all will look better. but it's still lovely!
		</review:text>
	</rdf:description>
	<rdf:description rdf:about="http://www.blogskins.com/info/150260">
		<dc:date xmlns:dc="http://purl.org/dc/terms/"
		rdf:datatype="http://www.w3.org/2001/XMLSchema#dateTime">20080421</dc:date>
	</rdf:description>
	<rdf:description rdf:about="http://www.blogskins.com/info/150260">
		<review:reviewer xmlns:review="http://purl.org/stuff/rev#"
		rdf:resource="http://www.blogskins.com/me/screamingtragic"/>
	</rdf:description>
	<rdf:description rdf:about="http://www.blogskins.com/me/screamingtragic">
		<rdf:type rdf:resource="http://www.w3.org/2006/vcard/ns#VCard"/>
	</rdf:description>
	<rdf:description rdf:about="http://www.blogskins.com/me/screamingtragic">
		<vcard:fn xmlns:vcard="http://www.w3.org/2006/vcard/ns#">screaming!tragic</vcard:fn>
	</rdf:description>
	<rdf:description rdf:about="http://www.blogskins.com/me/screamingtragic">
		<vcard:url xmlns:vcard="http://www.w3.org/2006/vcard/ns#"
		rdf:resource="http://www.blogskins.com/me/screamingtragic"/>
	</rdf:description>
	<rdf:description rdf:about="http://www.blogskins.com/info/150260">
		<rdfs:seealso xmlns:rdfs="http://www.w3.org/2000/01/rdf-schema#"
		rdf:resource="http://www.blogskins.com/info/150260"/>
	</rdf:description>
	<rdf:description rdf:about="http://www.blogskins.com/info/150260">
		<dc:subject xmlns:dc="http://purl.org/dc/terms/"
		rdf:resource="http://www.blogskins.com/info/150260"/>
	</rdf:description>
	<rdf:description rdf:about="http://www.blogskins.com/info/150260">
		<rel:review xmlns:rel="http://search.yahoo.com/searchmonkey-relation/"
		rdf:resource="http://www.blogskins.com/info/150260"/>
	</rdf:description>
	<rdf:description rdf:about="http://www.blogskins.com/info/150260">
		<rdf:type rdf:resource="http://purl.org/stuff/rev#Review"/>
	</rdf:description>
	<rdf:description rdf:about="http://www.blogskins.com/info/150260">
		<review:title xmlns:review="http://purl.org/stuff/rev#">Tekno Love Song</review:title>
	</rdf:description>
	<rdf:description rdf:about="http://www.blogskins.com/info/150260">
		<review:text xmlns:review="http://purl.org/stuff/rev#">you need stars! I think
		the bg shouldn't fade off. Other than that, it's fabulous</review:text>
	</rdf:description>
	<rdf:description rdf:about="http://www.blogskins.com/info/150260">
		<dc:date xmlns:dc="http://purl.org/dc/terms/"
		rdf:datatype="http://www.w3.org/2001/XMLSchema#dateTime">20070903</dc:date>
	</rdf:description>
	<rdf:description rdf:about="http://www.blogskins.com/info/150260">
		<review:reviewer xmlns:review="http://purl.org/stuff/rev#"
		rdf:resource="http://www.blogskins.com/me/animate_"/>
	</rdf:description>
	<rdf:description rdf:about="http://www.blogskins.com/me/animate_">
		<rdf:type rdf:resource="http://www.w3.org/2006/vcard/ns#VCard"/>
	</rdf:description>
	<rdf:description rdf:about="http://www.blogskins.com/me/animate_">
		<vcard:fn xmlns:vcard="http://www.w3.org/2006/vcard/ns#">animate_</vcard:fn>
	</rdf:description>
	<rdf:description rdf:about="http://www.blogskins.com/me/animate_">
		<vcard:url xmlns:vcard="http://www.w3.org/2006/vcard/ns#"
		rdf:resource="http://www.blogskins.com/me/animate_"/>
	</rdf:description>
	<rdf:description rdf:about="http://www.blogskins.com/info/150260">
		<rdfs:seealso xmlns:rdfs="http://www.w3.org/2000/01/rdf-schema#"
		rdf:resource="http://www.blogskins.com/info/150260"/>
	</rdf:description>
	<rdf:description rdf:about="http://www.blogskins.com/info/150260">
		<dc:subject xmlns:dc="http://purl.org/dc/terms/"
		rdf:resource="http://www.blogskins.com/info/150260"/>
	</rdf:description>
	<rdf:description rdf:about="http://www.blogskins.com/info/150260">
		<rel:review xmlns:rel="http://search.yahoo.com/searchmonkey-relation/"
		rdf:resource="http://www.blogskins.com/info/150260"/>
	</rdf:description>
	<rdf:description rdf:about="http://www.blogskins.com/info/150260">
		<rdf:type rdf:resource="http://purl.org/stuff/rev#Review"/>
	</rdf:description>
	<rdf:description rdf:about="http://www.blogskins.com/info/150260">
		<review:title xmlns:review="http://purl.org/stuff/rev#">Tekno Love Song</review:title>
	</rdf:description>
	<rdf:description rdf:about="http://www.blogskins.com/info/150260">
	<review:text xmlns:review="http://purl.org/stuff/rev#">hmm.. the bg shouldn't
	be white. should continue the pattern thingy... plus, the picture somehow
	doesn't appeal to me..</review:text>
	</rdf:description>
	<rdf:description rdf:about="http://www.blogskins.com/info/150260">
		<dc:date xmlns:dc="http://purl.org/dc/terms/"
		rdf:datatype="http://www.w3.org/2001/XMLSchema#dateTime">20070713</dc:date>
	</rdf:description>
	<rdf:description rdf:about="http://www.blogskins.com/info/150260">
		<review:reviewer xmlns:review="http://purl.org/stuff/rev#"
		rdf:resource="http://www.blogskins.com/me/madeline_tn"/>
	</rdf:description>
	<rdf:description rdf:about="http://www.blogskins.com/me/madeline_tn">
		<rdf:type rdf:resource="http://www.w3.org/2006/vcard/ns#VCard"/>
	</rdf:description>
	<rdf:description rdf:about="http://www.blogskins.com/me/madeline_tn">
		<vcard:fn xmlns:vcard="http://www.w3.org/2006/vcard/ns#">madeline_tn</vcard:fn>
	</rdf:description>
	<rdf:description rdf:about="http://www.blogskins.com/me/madeline_tn">
		<vcard:url xmlns:vcard="http://www.w3.org/2006/vcard/ns#"
		rdf:resource="http://www.blogskins.com/me/madeline_tn"/>
	</rdf:description>
	<rdf:description rdf:about="http://www.blogskins.com/info/150260">
		<rdfs:seealso xmlns:rdfs="http://www.w3.org/2000/01/rdf-schema#"
		rdf:resource="http://www.blogskins.com/info/150260"/>
	</rdf:description>
	<rdf:description rdf:about="http://www.blogskins.com/info/150260">
		<stag:tagged xmlns:stag="http://semantictagging.org/ns#"
		rdf:nodeID="id1565449429"/>
	</rdf:description>
	<rdf:description rdf:nodeID="id1565449429">
		<rdf:type rdf:resource="http://semantictagging.org/ns#Tag"/>
	</rdf:description>
	<rdf:description rdf:nodeID="id1565449429">
		<stag:label xmlns:stag="http://semantictagging.org/ns#">love</stag:label>
	</rdf:description>
	<rdf:description rdf:about="http://www.blogskins.com/info/150260">
		<stag:tagged xmlns:stag="http://semantictagging.org/ns#"
		rdf:nodeID="id1573342356"/>
	</rdf:description>
	<rdf:description rdf:nodeID="id1573342356">
		<rdf:type rdf:resource="http://semantictagging.org/ns#Tag"/>
	</rdf:description>
	<rdf:description rdf:nodeID="id1573342356">
		<stag:label xmlns:stag="http://semantictagging.org/ns#">song</stag:label>
	</rdf:description>
	<rdf:description rdf:about="http://www.blogskins.com/info/150260">
		<stag:tagged xmlns:stag="http://semantictagging.org/ns#"
		rdf:nodeID="id1573073203"/>
	</rdf:description>
	<rdf:description rdf:nodeID="id1573073203">
		<rdf:type rdf:resource="http://semantictagging.org/ns#Tag"/>
	</rdf:description>
	<rdf:description rdf:nodeID="id1573073203">
		<stag:label xmlns:stag="http://semantictagging.org/ns#">tekno</stag:label>
	</rdf:description>
	<rdf:description rdf:about="http://www.blogskins.com/info/150260">
		<stag:tagged xmlns:stag="http://semantictagging.org/ns#"
		rdf:nodeID="id1572116068"/>
	</rdf:description>
	<rdf:description rdf:nodeID="id1572116068">
		<rdf:type rdf:resource="http://semantictagging.org/ns#Tag"/>
	</rdf:description>
	<rdf:description rdf:nodeID="id1572116068">
		<stag:label xmlns:stag="http://semantictagging.org/ns#">cocorosie</stag:label>
	</rdf:description>
</rdf:RDF>
</searchmonkey_rdf>
<size>18514</size>
<title>BlogSkins.com Skin Info - Tekno Love Song</title>
<url>http://www.blogskins.com/info/150260</url>
</result>

Related articles by Zemanta

VN:F [1.7.5_995]
Rating: 0.0/5 (0 votes cast)
VN:F [1.7.5_995]
Rating: 0 (from 0 votes)

2 Responses to “Build a semantic Search Engine with BOSS”

Feed for this Entry Trackback Address
  1. 1 Atul
    February 3, 2010 at 3:34 pm

    If I search for “capitol grille” on http://search.yahoo.com

    It gives me the result with Microformat and without Microformat .

    Check the below Result which is with microformat returning the Adress and Image and the Ratings :

    Capitol Grille – Downtown – Nashville, TN
    User Reviews|Photos|Write Review•Yelp Rating: 5 reviews
    •Address: 231 6th Ave N, Nashville, TN 37219, USA
    •Phone: (615) 244-3121
    http://www.yelp.com/biz/capitol-grille-nashville

    Please inform me how can we do this microformat returning the Adress and Image and the Ratings .

    I am trying

    http://boss.yahooapis.com/ysearch/web/v1/Capitol+Grille+searchmonkeyid:com.yahoo.page.uf.hreview?appid=&format=xml&start=0&count=15&view=keyterms%2Csearchmonkey_feed%2Csearchmonkey_rdf

    But it is not giving me any result .

    PLEASE PLEASE HELP ME

    Regards
    Atul

    UN:F [1.7.5_995]
    Rating: 0.0/5 (0 votes cast)
    UN:F [1.7.5_995]
    Rating: 0 (from 0 votes)
  2. 2 Ted
    February 3, 2010 at 7:15 pm

    Hi Atul,
    I sent a message to the BOSS mailing list, but try the simplified filters listed in this blog post: http://www.bosshacks.com/searchmonkey-data-in-boss.html

    However, I noticed only two results (from Yahoo! Local) for that restaurant. I’m not sure why it isn’t pulling in the Yelp responses.

    UA:F [1.7.5_995]
    Rating: 0.0/5 (0 votes cast)
    UA:F [1.7.5_995]
    Rating: 0 (from 0 votes)

Leave a Reply

Powered by WP Hashcash