S3 (Simple Storage Service) – Amazon and Libraries

Have you heard of Amazon’s s3 (Simple Storage Service)? From the site:

Amazon S3 is “storage for the Internet” with a simple Web services interface that can be used to store and retrieve any amount of data, at any time, from anywhere on the Web.

It’s one of Amazon’s newer web services. At .15 cents/gig of storage, it’s a pretty cheap option. Caveat emptor: S3 is intended for developers as an option for storage that can be queried with SOAP and REST web services, so they also get you for network traffic at .25 cents/gig. I wasn’t able to find anything in the fine print about checksum routines and the integrity of the objects, but I’m assuming backups and error checking are part of the Amazon routine. (Update from the horse’s mouth: found this thread in the forums which talks about Amazon’s data protection routines. It’s reassuring…)

Can the library use this? I think so. Even with the mentioned caveats, in the end you are looking at taking the server management side out of the equation. That’s pretty liberating for the small digital shops that our libraries are. At work, we’re experimenting with using the service to store some of our master digitization objects. I mentioned that this was an experiment, right? We’ve got some objects on the S3 servers and are looking into building a web interface that will allow our Special Collections staff to pull down master files when they receive requests from patrons. We’re also working with a campus entity to store media files on S3 and then building a search interface to query S3 for the data. It’s all a work in progress, but something to consider. I can tell you that my library and university will never have the infrastructure or access to a network cloud like Amazon’s. That’s not a knock; them’s just the facts.

(Sidebar: If you’re interested in web services, think about browsing around the Amazon Web Services Developer Connection. Lots of code examples, “howtos” and discussion to get you thinking about web service applications. Don’t be afraid to get you hands dirty and make some mistakes. It’s the only way to learn.)


Anatomy of a Function

It’s been a little while since my last post. My recent work schedule and Turkey Day played a part in that. I’ve been working .com hours on a super cool project. (I mentioned the TERRA group in an earlier post.) I’ve learned so much in the last couple of weeks. It’s amazing what a hard deadline and a shifting set of requirements will do to your web programming skills. All the hard work is about to bear fruit as as the new TERRA web site is about to go live. Have a look at a different kind of digital library.

But, that’s not the point of this post… I wanted to share a little code that made some data conversion very simple over the course of the TERRA project. It’s a simple little php function that converts a MySQL timestamp into an RFC 822 date format (For the project, we stored the item update fields as timestamps and then converted them when we generated our various XML feeds. RFC 822 or RFC 2822 are necessary for valid feeds.) Here’s the php function in all its glory:

//function converts mysql timestamp into rfc 822 date
function dateConvertTimestamp($mysqlDate)
{
$rawdate=strtotime($mysqlDate);
if ($rawdate == -1) {
$convertedDate = ‘conversion failed';
} else {
$convertedDate = date(‘D, d M Y h:i:s T’,$rawdate);
return $convertedDate;
}
}
//end dateConvertTimestamp

You call the function by including it on the page and using the following code:

$newPubdate = dateConvert(“$stringToConvert”);
echo $newPubdate;

Where $stringToConvert would be any MySQL timestamp value that needs conversion.

In the end a string like this “2005-05-17 12:00:00″ looks something like this “Tue, 17 May 2005 12:00:00 EST”. You could also reverse the conversion using this php function:

//function converts rfc 822 date into mysql timestamp
function dateConvert($rssDate)
{
$rawdate=strtotime($rssDate);
if ($rawdate == -1) {
$convertedDate = ‘conversion failed';
} else {
$convertedDate = date(‘Y-m-d h:i:s’,$rawdate);
return $convertedDate;
}
}
//end dateConvert

NOTE: If/when you copy and paste the above code, make sure all ” (double quotes) and ‘ (single quotes) are retyped. WordPress is doing a number on the proper format.

I just wanted to share the wealth a bit. If you’ve got questions or suggestions, don’t be shy about dropping a comment. I’ll be home in Wisconsin for the next several days, but I’ll have limited internet access there.  I’ll try to answer questions if they arise.


Dueling Ajax – couple of articles

I’ve been a bit of the Ajax poster boy lately. Two pieces that I wrote for library audiences have just been published.

Building an Ajax (Asynchronous JavaScript and XML) Application from Scratch.” Computers in Libraries 26, no. 10 (November/December 2006).

Ajax (Asynchronous JavaScript and XML): This Isn’t the Web I’m Used To.” Online 30, no. 6 (November/December 2006)
uri: http://www.infotoday.com/Online/nov06/Clark.shtml

Both articles fall into the “introductory” mode, although the CIL article walks you through a proof of concept Ajax page update script (mentioned in an earlier post…). I want to be clear: I’m not an Ajax evangelist. I find the suite of technologies that make Ajax go intriguing and the improvements that the Ajax framework can make to some library applications are worth learning about and applying. I tried to point out the good and the bad. Although, it is a four letter word…

I did want to mention a couple of books that were really helpful in getting me up to speed with the Ajax method.

Ajax in Action by Dave Crane, Eric Pascarello and Darren James

DHTML Utopia Modern Web Design Using JavaScript & DOM by Stuart Langridge

(Click on the book covers if you are into book learnin’ and want to browse the Amazon records.) Dig in and discover (or rediscover) some of the possibilities when you put Javascript to work in your apps.


What’s up with Ruby on Rails?

Yeah, I’ve been playing around with Ruby on Rails and I gotta’ say it’s pretty slick. Where should I start? Intuitive functions, easy mapping of database table relationships, simple templating (using layout views), MVC (model, view, controller) application architecture…

The MVC structure is nice, but it took me a little time to realize where each piece of the application lived. It also has some data model quirks – all tables must use a plural format naming scheme (e.g., table = users), foreign keys must have a format of “name_id” (e.g., users table foreign key = user_id), column names created_on and updated_on will automatically be populated correctly, etc. You get a sense of these rules as you start to use the framework. The documentation should catch up with these problems eventually. (I hope.)

So it has some limitations, but the real advantage is the generate scaffold command that reads data tables and maps simple html forms and pages for all CRUD (Create, Read, Update, and Delete) functions of a database web app. And this is where it seems to leap ahead of php – you can have a functioning dynamic web app in about a half hour. It’s bare bones, but it works. Lots of tutorials and introductions out there. Amy Hoy’s Really Getting Started in Rails is one of the better ones. If you are into book learnin’, Agile Web Development with Rails: A Pragmatic guide is a good place to start. Check it out at http://worldcatlibraries.org/wcpa/isbn/097669400X.


Follow

Get every new post delivered to your Inbox.