[COLUG] JAQL, JSON, Hadoop and the like..

[Tom Hanlon]

Colug,

Looking to perhaps start a general discussion.

Does anyone here do any work with

- JSON (Java Script Object Notation)-
A way to define arrays, lists and the like and access them through  
Javascript ?
I assume some of the AJAX developers mess around with this, either  
explicitly or through a collection of tools.

-JAQL-
Has anyone using JSON looked at JAQL ?
http://www.jaql.org/

-HADOOP-
Does anyone mess with any of the frameworks for building large  
scalable web apps. Such as
Hadoop, or Map/Reduce
http://hadoop.apache.org/core/

Hadoop was actually spawned from the Lucene text search tool from the  
apache software foundation. Does anyone mess with Lucene.


-Lucene-
Text search tool. More robust than something like a plaintext index  
on a DB text column. DB plain text, at least in MySQL does not allow  
building of phrases. No word order is kept in the Btree built on the  
words. Once we talk about Lucene we should also talk about
http://lucene.apache.org/java/docs/

-Sphinx-
Text indexing tool, uses MySQL but is more complete than a fulltext  
index.
http://www.sphinxsearch.com/

-Map/Reduce-
This is a Google implementation to deal with large datasets over  
clusters of machines.
http://labs.google.com/papers/mapreduce.html

Come to think of it once we look at the Google way of doing things we  
should also consider the amazon way of doing things.. Has anyone  
looked at or played with

-simpledb-
http://www.amazon.com/b?ie=UTF8&node=342335011
A tool for querying name-value pairs across the google cloud or  
whatever they call there hosted architecture.

Reasons...
Well I teach classes to some folks that are using these tools and/or  
would like to know more about them. Since I "teach" more than I "do"  
I struggle to get practical experience. I hear lots of buzz and I  
know a few things here and there but when it comes to sorting out the  
newer stuff on the horizon, well I look to fellow list members for  
advice and opinions. If you have messed with some of these  
technologies I would like to know why, how and if it worked out for  
you ?

The JAQL stuff looked interesting to me as a way of simplifying/ 
extending the relational model and or simplifying it to a shared  
architecture. The bigger database implementations tend to lose most  
of the fancy features as they optimize for speed. transactions,  
triggers, stored procedures, consistency, well they all get thrown  
out or pushed to the application or modified/muddied in some ways as  
you optimize for speed, and your data becomes more and more like a  
collection of name/value pairs that you need to get to FAST. So the  
JAQL stuff caught my eye. I might try and play around with it or  
explore.

The amazon simpledb stuff seems to be leaning in that direction.

As you look at big web implementations it is interesting to compare  
and contrast the amazon way vs the google way. There are other ways  
of course and mixtures but those are obviously two good starting points.


--
Tom


--
Tom