[+]Twitter Add to Google! Add to My Yahoo! Subscribe with Bloglines

Archived Posts from “API”

MapReduce Database Views as URI Resources

08

September

I believe we’re entering the Functional Reactive Programming paradigm shift. At least I hope we are.

Behaviors that update views on data only when that data changes.

Named resources ala URIs that expose those views, thereby exposing that data through APIs ala REST, ala CouchDB.

Views that can then be dynamically compiled into pages using transclusion (the embedding of externally identified, content) and functional reactive programming client behaviors ala Flapjax or more conventionally Redux. Views that can be programmed live using FRP techniques ala SuperGlue.

Flat tables ala CouchDB, Googles BigTable, Hadoop? and Vertica, storing large datasets viewed with client-driven(pay?) behavioral views that form customised APIs. Behaviour Orientated Architecture. BOA.
:)
Forget about maintaining the APIs and worry about maintaining the data and the ability to view that data. ie. computational resources to generate and distribute those views, as well as uptime and security permissions for that data. HTTP Digest, view access permissions? [Insert Identity Layer Here] Better yet, automated content-centric networking, encrypted packet creation of those views as named data that can be pushed into the cloud.

Next we take it to the architecture level and synchronous reactive programming. Then we get functional reactive AI systems.
;)


HTTP URI Methods Extension & REST?

21

November

Update: http://www.w3.org/Protocols/PEP/ does what I'm looking for. :)
I'm throwing this half-baked idea out there for some feedback to see how feasible it might be.
It's basically extending HTTP with directional URI-namespace methods. URI specfic methods could be looked up by a client with HTTPs OPTION method. The returned Allow: response then used to determine what API methods can be performed on any particular resource. URI Namespacing them would allow anyone to create their own documented API that could be looked up manually or auto-discovered and documented with crawlers.

Method naming would indicate in which direction messages are passed with just the .POST and .GET (Input/Output) suffixes? doing away with PUT and DELETE and other keyword methods in favour of directional URI name space methods. Stateless producer/consumer.

eg:

Resource Methods Introspection:

Request:

OPTIONS /people/person HTTP/1.1
Host:example.com

Response:

Allow: GET, POST, OPTIONS, xmlns.com/foaf/0.1/Person.GET, xmlns.com/foaf/0.1/Person.name.POST
In this example the foaf:name documentation section of the spec could be returned on sending a GET to the "http://xmlns.com/foaf/0.1/Person.GET" URI. Or the "xmlns.com/foaf/0.1/Person.GET" method could be called on the resource that returned it, for example:

Request:

xmlns.com/foaf/0.1/Person.GET /people/person HTTP/1.1
Host: example.com

Response:

<foaf:Person>

<foaf:name>Craig</foaf:name>
<foaf:homepage rdf:resource="http://example.com/blog/"/>

</foaf:Person>

Say you wanted to update the foaf:name element for http://example.com/people/person you might use a method that looks like this:

Request:

xmlns.com/foaf/0.1/Person.name.update.POST /people/person HTTP/1.1
Host: example.com

<foaf:name>ENDOVER Craig</foaf:name>

Response:

HTTP/1.1 200 OK
Date: Mon, 20 Nov 2006 14:30:17 GMT

<foaf:name>ENDOVER Craig</foaf:name>

Delete a Person like so:

xmlns.com/foaf/0.1/Person.delete.POST /people/person HTTP/1.1

Performing a straight GET on http://example.com/people/person could return an entire blog whereas the foaf:name method call just returns a the blog owners foaf.

Consider other APIs and a different syntax:

microformats.org/URImethods.hCard.FN.POST /people/person HTTP/1.1
microformats.org/URImethods.hCard.URL.POST /people/person HTTP/1.1
w3c.org.SPARQL.1_0.endpoint.GET /people/person HTTP/1.1
w3c.org.OWL.sameAS.GET /people/person HTTP/1.1
w3c.org.OWL.sameAS.POST /people/person HTTP/1.1

Annote the semantic web?

One thought is that GET/POST may not be ideal. eg. Benjamin Carlyle's SENA internet draft comes to mind

Consumer:
soundadvice.id.au.SENA.SUBSCRIBE.POST
soundadvice.id.au.SENA.NOTIFY.GET
soundadvice.id.au.SENA.NOTIFYPATCH.GET
soundadvice.id.au.SENA.EXPIRE.GET

Producer:
soundadvice.id.au.SENA.SUBSCRIBE.GET
soundadvice.id.au.SENA.NOTIFY.POST
soundadvice.id.au.SENA.NOTIFYPATCH.POST
soundadvice.id.au.SENA.EXPIRE.POST

INBOX / OUTBOX may be a better choice?

Consumer:
soundadvice.id.au.SENA.SUBSCRIBE.OUTBOX
soundadvice.id.au.SENA.NOTIFY.INBOX
soundadvice.id.au.SENA.NOTIFYPATCH.INBOX
soundadvice.id.au.SENA.EXPIRE.INBOX

Producer:
soundadvice.id.au.SENA.SUBSCRIBE.INBOX
soundadvice.id.au.SENA.NOTIFY.OUTBOX
soundadvice.id.au.SENA.NOTIFYPATCH.OUTBOX
soundadvice.id.au.SENA.EXPIRE.OUTBOX

This all seems feasilble to me but I'm only just getting my feet wet with HTTP, so feedback would be greatly appreciated.

Craig.


On Open Data Storage API’s with Identity

17

July

I've been thinking about data distribution and the open API model services like Amazon S3 offer. The more I think about it, the more I see a need for an identity system with integrated online data storage. I call this an i-drive API spec. Something any hosting provider could implement.
Here's how I see it work; first a standard form of data storage API be developed with all the bells and whistles for users to secure their data and access though a standard protocol. Possibly different levels of API security spec too. Not everyone wants to store secure data and not every provider has the resources to implement high security. I'd like the option to be able to store my photo's/music/data on my own server for example. Other sensitive data on secure services. Anyway, users are then given the option of where they choose to store that data when using identity broker services. The Yadis identity service discovery system then shares that i-drive information with i-drive enabled apps. The apps can cache if the API lets it. (much like I believe XDI link contracts work). This way, anyone can store specific data anywhere that provides an i-drive service, even multiple services for the same data, eg. backup and redundancy. Some people might want that data all in one place. Me, I like diversity and choice.

As I understand it XDI will do some(all?) of this for i-names. A full blown i-broker implementation I'm guessing? More to read…

Ideally I'd like to be able to grab some open source i-drive code, dump it on my on own web host, tell my Yadis supported identity account where my i-drive is located and what it's to be used for and all the rest between my web apps and i-drive is taken care of for me. Additionally I could then implement my own open source OpenID(or i-names) broker with i-drive on my server then and start using it and encouraging sites like flickr to support it. It could all be built into my blogware. That'd be sweet. I could even have an i-drive service running on my operating system keeping sync (if my bandwidth costs weren't so high).

Using this service I could subscribe to all sorts of data that gets added to my i-drive and sync'd to specific i-drive devices (TV/Music Player). YouTube on my real tube.

Anyone know of efforts around other than XDI looking to do something like this?

Update: Another thought. Triggered while reading Mark Cuban. Make each and everyone of these i-drives BitTorrent capable.

Update: A bare bones API extendable to include data manipulation extensions for web apps to make use of. eg. attach advert clip to a podcast, resize an image, seek to a video or comment location inside media, etc. WebDAV?

Update: I found this great post by Christine when looking for another of hers I'd read previously. It covers the need for decentralized data access. Well worth a good read.


How to Build a Podcast/Vlog Network 101

17

July

I've been following Cameron Reilly's progress with ThePodcastNetwork for a while now. Listening to his current woes, hoping things pick up soon for him. His request for ideas had me thinking and I've decided to pitch some of my ideas in. Feel free to steal them, I have much bigger ones in the works. :) Anyway here are my suggestions and thought I'd publish them for all to see and debate.

I'll start by outlining the podcast hosting (or video log for that matter) business model as I see it. Advertising. With more and more services like Amazon's S3 our likely future, I see the bottom eventually falling out of mere hosting unless you make the hardware or maintain it. Still you can cream a little off the top using it if you like, but for me the future is all advertising and subscriptions with hosting a side dish unless your a data centre.

Now Cameron more than anyone knows that advertising for his medium will most likely come through embedded sponsorship in podcasts as well as targeted on his site. Thus the more listeners and distribution, the better exposure for all involved. In my mind therefore building brand reputation while building distribution channels are the key to this medium. So, first we build on those. I use Cameron's business as my case study here. So here are my suggestions:

1. Build a better distribution model.

Make it easy for anyone to access and share your data.

The first and easiest thing Cameron could do is make each and every podcast easily sharable on anyone's site with a custom and branded flash player(with fallback) they can drop in, ala Youtube.

Optionally, a player, that on mouse hover or excerpt button click, displays either a text excerpt or transcript of the cast. (I've heard anecdotal evidence that transcripts next to casts improve listens by up to 225%.) Note: I'd do it with display:none javascript, not ajax to help the search engines find it. Let's hope RSS readers of the future support display:none or some other safe means of embedding metadata, aye. (Comments on alternative methods welcome)

So for easy copying and pasting I see the implemenation best done through and interface something like Live Clipboard's.

Live Clipboard Cut, Copy, Paste, Delete

Thus we display Live Clipboard scissors on objects considered sharable. With our podcast example, on hovering those scissors would display your options for that content. In this case clicking Copy simply copies a piece of HTML to the users clipboard containing an object element marked up. ie. the podcast player <object> with podcast uri and description.

Note: This example doesn't utilise the 'Live' of Live Clipboard, but you get the idea. BTW, to anyone looking to implement Live Clipboard, please, on selecting a method of an object the user should be presented with something like a tooltip instructing them what has just happened! eg. "Success! HTML Object copied to your operating system clipboard. Now, go paste it somewhere!" or "Success! Live Clipboard XML Object copied to your operating system clipboard. Perhaps on hovering objects the type of content to be copied is displayed. eg. replace Copy (CTRL-C) in the figure above with Copy HTML Object (CTRL-C) or Copy Live Clipboard Object (CTRL-C)

Now, back to distributing our podcasts.

I'd also suggest branded widgets of feeds copied and pasted in similar fashion.

Taking this further an open API service for database access to casts and their metadata as well as data processing services. This however requires infrastructure and one central content management and hosting system. So build one(if and when you can afford to!). And Cameron, I know I'd have a 2web widget on my blog if you had one. :)

Next for distribution, another interesting path I see this model taking is with distributed metadata creation services. Check out what the blip.tv guys are working on with mirrorplay (not up yet) and read about video vertigo (user and password are in the popup authentication) and where metadata sharing for distributed media is headed.

There's an opportunity here for someone to build an open API that lets developers build podcasting software into their own, have those podcasts hosted on the API providers servers and allow collaboration with advertisers through it. Or just a service that streams the process of adding ads to the media and then to the clients servers. It's complicated to explain but if your interested, have a look at the eventful aka EVDB model. I think it's kinda similar but in their case used for events and I hear purchasing tickets.

And BTW, while your at it creating widgets, don't forget to add digg and delicious buttons to each! And comments… You could even include an area in the widget for displaying tags and tagging that very podcast using AJAX! No need then for mirrorplay? Maybe. Maybe not. Something worth considering is that not everyone lets you embed scripts… alternative: flash… *grumble*

Additionally you should write or purchase rights or licensing to podcast aggregating software for downloading content locally and consumuing it and feeding the playlist back to your database. Don't forget to brand it all.

Don't forget to bundle your service with other hardware. Mobile devices especially are the media players of the future.

2. Build a solid brand image.

Produce quality content under your logo stamp. Seek out existing podcasters(or vloggers, whatever floats your boat) with a proven track record and bring em aboard your stamp. Podtech did this with Scoble, Scoble did this with GETV

Make podcasts about the people. Interviews, exclusives… I love those. Encourage that in your producers. Everyone loves to hear from experts in their field. Experts market themselves, experts therefore market you if you can get em.

3. Design and marketing.

Now I'm basing this off Camerons site and his homepage is a little bland. Get a good designer Cam, make it fun and build upon the brand in the design. In Camerons case, get rid of the whitespace at the top and make the TPN logo stand out. Add some mr.sheen polish to it and develop a 'look & feel' synonymous with TPN. Make play buttons using the logo to use there and in widgets? Something with personality… :)

Add a "featured show(s)" section with clips and play buttons. Add a short playable promo clip for all the shows and latest shows on there so people can get a feel for the hosts quickly. We're all time poor so please don't make me listen through ads or fancy long intros on those promo clips. Make the search feature and categories more visible. Make the site fun and idiot proof.

Maybe add sound effects on hovering items. Make promo's automatically play on hovering… Grab people's attention. Just never play music on page loading! That's a heinous usability crime! I curse you myspace.

Also add features that will entice people back. Let people tag and mark casts as favorites for later referral. Capture that attention data on what each user plays/tags/does on your site. Create a social network of tags and "who tags like me", "who likes listening to what I do" network ala last.fm. Create widgets of this data for the users. Use the data to recommend other items, including advertising. Create widgets for those.
Ultimately, you'll be reaching your users through distribution channels. Get that right. Make it so that people can easily find your content on your site to help it go viral. Add feeds and widgets on everything. Even categories, peoples listening lists, top downloaded… basic SEO.
4. Education

Drive traffic through education. Create a podcasting tutorial blog(and cast) to educate people. Let the TPN producer crew write it, everyone has a howto podcast story right? Call it the Digital Podcasting School ala Darren Rowse and his Digital Photography School.

5. Advertising

First, create some. Your own! Build a service that can attach advertising automatically to media. Create your own advertising first for podcasts in your network to test the system and attract advertisers that listen. Add an advertise with us option to your widgets. Failing those, hire someone to find you advertisers. Or something. Advertising hurts my brain.

6. Product Affiliates

Create a "cool audio tool" directory using amazon and other affiliates who'd like to advertise their wares to podcasters/users through you. This would fit well on the education blog and next to podcasts. Include hardware/software recommendations, reviews, etc. Track it all with stats.

7. Service Affiliates

"Advertisment FREE" podcasts for paying clients or subscribers.

8. Statistics

Watch them and learn.

9. Brainstorm

Put all the content producers on a mailing list if they're not already. Bounce ideas in it with them all. Create a community wiki to document all those ideas. They'll want to help, they're making a cut of the money for producing the content…

10. Get good developers to build it all

So you can sit back, feet in the air and let someone else stress out. I should also talk about networking, blah, blah, but I'm no good at that. Just do it. You'll figure it out with all that traffic and attention you'll be getting from people.

In conclusion:

  • Build an API
  • Build your own service on top of that API
  • Open your API and service to the Public

The future is in API's and widgets(smart widgets and apps that talk back to your server with attention data).


PeopleAggregator - Open Social Networking

28

June

Today, Marc Canter took the wrapper off PeopleAggregator

PeopleAggregator is a Social Network with a difference. You can build your own site and network of people, groups, and media with it!

While on first impressions the user interface and design is hideous, it looks very promising. Communities will love the APIs!

Here are some of the features:

  • AJAX drag-n-drop content sections on your personal peofile page that displays your relations (friends, colleges, etc), a photo, your flickr, delicious, networks you belong to and a link list. There's a personal info section you can fill out and place access restrictions on who can see what parts of that information. Nobody, everybody and immediate relations privacy measures. There's the ability to upload your CV and list your professional info as well as snippets of text area information for you to fill in the details about you.
  • You can join groups and browse members content
  • Upload Media: Images (up to 500kb), tag it and describe it while setting image access and what album you place it in. Audio or Video (up to 3MB)
  • Blog and create content either a post, audio, images, video, events, reviews, people or groups.
  • Creating groups allow you to create links to external group spaces or ones you've made in your people aggregator network.
  • People, this item lets you describe peoples details.
  • Review all sorts of different standard types.. eg: local services, events, bar/clubs, books…
  • Event creation
  • All using structureblogging formats it appears.
  • A simple manager to manage your content.
  • A search to search that content
  • A gallery of your Images/Audio/Video, you can also view friends media or groups media
  • Clicking on an audio loaded windows movie player, Video loaded embedded Quicktime for .mov (without any video)
  • There's a people search, by details though not very detailed. Age isn't listed.
  • You can create your own groups and invite people.
  • Create a network for anyone to join and create groups inside that network. Media too, basically aggregate member content into the network. In other words; create social networks inside the people aggregator social network. This would have to be the best feature, allowing communities to build their own homes and share their content with one another. It's just a pity the site is ugly and the content creation too complicated and long-winded in a lot of areas. This is where the API's might come in handy to design your own.

If you'd like an invite, leave a comment.


Recent Links

Recent Links

-->
Recent Comments
  • Craig Overend: Fixed, thanks Josh. English and explaining myself clearly has never been a strength of mine. Glad you...
  • Josh: Hey, just wanted to point out it should be "you're", as in "you are". Otherwise, wow - very in depth post....
  • Joe Andrieu: Craig, As I've mentioned elsewhere, user-driven is a solid improvement over user-centric, both...
  • Niall Kennedy: Asking the site visitor to opt-in would defeat the purpose in my particular case. I am trying to...
  • Craig Overend: Without qualifying yourself I find that comment facetious. If your playing on my use of the term...