Showing posts with label nosql. Show all posts
Showing posts with label nosql. Show all posts

Thursday, 11 July 2013






Introduction


  • Apache CouchDB, commonly referred to as CouchDB, is an open source database that focuses on ease of use and on being "a database that completely embraces the web".
  • It is a NoSQL database that uses JSON to store data, JavaScript as its query language using MapReduce and HTTP for an API.
  • One of its distinguishing features is multi-master replication.
  • Couch is an acronym for cluster of unreliable commodity hardware
  • A CouchDB database lacks a schema, or rigid pre-defined data structures such as tables. Data stored in CouchDB is a JSON document(s).
  • The structure of the data, or document(s), can change dynamically to accommodate evolving needs.
  • CouchDB also offers a built-in administration interface accessible via web called Futon.

Main features



  • Document Storage:
    • CouchDB stores data as "documents", as one or more field/value pairs expressed as JSON. Field values can be simple things like strings, numbers, or dates; but ordered lists and associative arrays can also be used. Every document in a CouchDB database has a unique id and there is no required document schema.


  • ACID Semantics:
    • CouchDB provides ACID semantics. It does this by implementing a form of Multi-Version Concurrency Control, meaning that CouchDB can handle a high volume of concurrent readers and writers without conflict.


  • Map/Reduce Views and Indexes:
    • The stored data is structured using views. In CouchDB, each view is constructed by a JavaScript function that acts as the Map half of a map/reduce operation. The function takes a document and transforms it into a single value which it returns. CouchDB can index views and keep those indexes updated as documents are added, removed, or updated.


  • Distributed Architecture with Replication:
    • CouchDB was designed with bi-direction replication (or synchronization) and off-line operation in mind. That means multiple replicas can have their own copies of the same data, modify it, and then sync those changes at a later time.


  • REST API:
    • All items have a unique URI that gets exposed via HTTP. REST uses the HTTP methods POST, GET, PUT and DELETE for the four basic CRUD (Create, Read, Update, Delete) operations on all resources.


  • Eventual Consistency:
    • CouchDB guarantees eventual consistency to be able to provide both availability and partition tolerance.


  • Built for Offline:
    • CouchDB can replicate to devices (like smartphones) that can go offline and handle data sync for you when the device is back online.


  • Schema-Free:
    • Unlike SQL databases, which are designed to store and report on highly structured, interrelated data, CouchDB is designed to store and report on large amounts of semi-structured, document oriented data. CouchDB greatly simplifies the development of document oriented applications, such as collaborative web applications.
    • In an SQL database, the schema and storage of the existing data must be updated as needs evolve. With CouchDB, no schema is required, so new document types with new meaning can be safely added alongside the old. However, for applications requiring robust validation of new documents custom validation functions are possible. The view engine is designed to easily handle new document types and disparate but similar documents.


Installation (Ubuntu)



  • Open terminal and execute
sudo apt-get install couchdb -y


  • If the aptitude/apt-get installation gives an error message then couchdb might not have access to its pid file.


  • Fix:
sudo chown -R couchdb /var/run/couchdb
  • Need to rerun the setup script:
sudo dpkg --configure couchdb




For Verifying CouchDB, check on Futon: http://127.0.0.1:5984/_utils/index.html

*For installation on other Operating Systems - http://wiki.apache.org/couchdb/Installation


CouchDB Java Client



  • Ektorp


  • JRelax


  • jcouchdb


  • DroidCouch


  • CouchDB4J


  • LightCouch



CouchDB4J Example



  • CouchDB4J is an updated Java library for CouchDB.
  • It handles the REST style calls to the CouchDB server behind the scenes, and give you a handle on the JSON objects directly.
  • CouchDB4J uses JSON-lib to handle mapping to/from JSON objects, which makes getting/setting properties on the objects very easy.
  • You can even map Java objects to JSON objects and back to make the process easier.
  • Example:


Structure:


CouchDBCompleteDemo.java



package com.couchdb.demo;

import java.io.BufferedReader;

import java.io.IOException;

import java.io.InputStream;

import java.io.InputStreamReader;

import java.util.List;

import java.util.Map;

import org.apache.http.HttpEntity;

import org.apache.http.HttpResponse;

import org.apache.http.client.ClientProtocolException;

import org.apache.http.client.methods.HttpGet;

import org.apache.http.impl.client.DefaultHttpClient;

import org.json.simple.JSONValue;

import com.fourspaces.couchdb.Database;

import com.fourspaces.couchdb.Document;

import com.fourspaces.couchdb.Session;

import com.fourspaces.couchdb.ViewResults;

public class CouchDBCompleteDemo {

   static Session dbSession ;

   static Database db;

   

   public static void main(String[] args) {

       String dbName = "foodb";

       createDatabase(dbName);

       

       saveDocument(getDocument("1", "Willian", "J2EECOE", "TL", "Android"));

       saveDocument(getDocument("2", "Joanne", "J2EECOE", "Developer", "Java"));

       saveDocument(getDocument("3", "suzzane", "J2EECOE", "Sr. Developer", "Java"));

       saveDocument(getDocument("4", "Harley", "J2EECOE", "Sr. Developer", "Java"));

       saveDocument(getDocument("5", "Julian", "J2EECOE", "Developer", "Java"));

       saveDocument(getDocument("6", "Peter", "J2EECOE", "Developer", "Java"));

       getTotalDocumentCount();

       viewAllDocuments();

       viewsDemo();

//        deleteDocument("6");

//        getTotalDocumentCount();

//        deleteDatabase(dbName);

       

   }

   public static void createDatabase(String dbName){

       dbSession = new Session("localhost", 5984);

       db = dbSession.createDatabase(dbName);

       if(db==null)

           db = dbSession.getDatabase(dbName);

   }

   public static Document getDocument(String id,String name,String group,String designation,String language){

       Document doc = new Document();

       doc.setId(id);

       doc.put("EmpNO", id);

       doc.put("Name", name);

       doc.put("Group", group);

       doc.put("Designation", designation);

       doc.put("Language", language);

       return doc;

   }

   

   public static void saveDocument(Document doc){

       try {

           db.saveDocument(doc);

       } catch (Exception e) {

       }

   }

   

   public static int getTotalDocumentCount(){

       int count = db.getDocumentCount();

       System.out.println("Total Documents: " + count);

       return count;

   }

   

   public static void deleteDocument(String id){

       Document d = db.getDocument(id);

       System.out.println("Document 1: " + d);

       db.deleteDocument(d);

   }

   

   public static void deleteDatabase(String dbName){

       dbSession.deleteDatabase(dbName);

   }

   public static void viewAllDocuments(){

       ViewResults results = db.getAllDocuments();

       List<Document> documentsList = results.getResults();

       if(documentsList!=null)

       {

           for(Document doc : documentsList)

           {

               System.out.println(doc.get("id") + " : " +  doc);

           }

       }

   }

   

   public static void viewsDemo(){

       if(db!=null)

       {

Document doc = null;

           try {

               doc = db.getDocument("_design/couchview");

           } catch (Exception e1) {

               doc = null;

           }

           try {

               if(doc==null)

               {

                   doc = new Document();

                   doc.setId("_design/couchview");

                   String str = "{\"javalanguage\": {\"map\": \"function(doc) { if (doc.Language == 'Java')  emit(null, doc) } \"}, \"java_and_se\": {\"map\": \"function(doc) { if (doc.Language == 'Java' & doc.Designation == 'SE')  emit(null, doc) } \"}}";

                   doc.put("views", str);

                   db.saveDocument(doc);

               }

           } catch (Exception e) {

           }

       }

       

       try {

           DefaultHttpClient httpclient = new DefaultHttpClient();

           

           HttpGet get = new HttpGet("http://localhost:5984/foodb/_design/couchview/_view/javalanguage");

           HttpResponse response = httpclient.execute(get);

           HttpEntity entity=response.getEntity();

           InputStream instream = entity.getContent();

           

           BufferedReader reader = new BufferedReader(new InputStreamReader(instream));

           String strdata = null;

           String jsonString = "" ;

           while( (strdata =reader.readLine())!=null)

           {

//                   System.out.println(strdata);

                  jsonString += strdata;

           }

           

           System.out.println("Json String: " + jsonString);

           Map<String, Object> jsonMap = getMapFromJsonString(jsonString);

           

           if(jsonMap!=null)

           {

               System.out.println("total_rows: " + jsonMap.get("total_rows"));

               System.out.println("offset: " + jsonMap.get("offset"));

               List<Map> rowsList = (List<Map>) jsonMap.get("rows");

               if(rowsList!=null)

               {

                   for(Map row: rowsList)

                   {

                       System.out.println("----------------");

                       System.out.println("Id: " + row.get("id"));

                       System.out.println("Value: " + row.get("value"));

                       System.out.println("Name: " + ((Map)row.get("value")).get("Name"));

                       System.out.println("_id: " + ((Map)row.get("value")).get("_id"));

                       System.out.println("Language: " + ((Map)row.get("value")).get("Language"));

                       System.out.println("EmpNO: " + ((Map)row.get("value")).get("EmpNO"));

                       System.out.println("Designation: " + ((Map)row.get("value")).get("Designation"));

                       System.out.println("Group: " + ((Map)row.get("value")).get("Group"));

                   }

               }

           }

       } catch (ClientProtocolException e) {

           e.printStackTrace();

       } catch (IllegalStateException e) {

           e.printStackTrace();

       } catch (IOException e) {

           // TODO Auto-generated catch block

           e.printStackTrace();

       }    

   }

   

   public static Map<String, Object> getMapFromJsonString(String jsonString){

       Map<String, Object> jsonMap = (Map<String, Object>) JSONValue.parse(jsonString);

       System.out.println("Json Map: " + jsonMap);

       return jsonMap;

   }
}

Thursday, 20 June 2013

Introduction


  • Amazon DynamoDB is a fully managed NoSQL database service that provides fast and predictable performance with seamless scalability.
  • Amazon DynamoDB automatically spreads the data and traffic for the table over a sufficient number of servers to handle the request capacity specified by the customer and the amount of data stored, while maintaining consistent and fast performance.
  • All data items are stored on Solid State Disks (SSDs) and are automatically replicated across multiple Availability Zones in a Region to provide built-in high availability and data durability.
  • You can launch a new Amazon DynamoDB database table, scale up or down your request capacity for the table without downtime or performance degradation, and gain visibility into resource utilization and performance metrics, all through the AWS Management Console.
  • With Amazon DynamoDB, you can offload the administrative burdens of operating and scaling distributed databases to AWS, so you don’t have to worry about hardware provisioning, setup and configuration, replication, software patching, or cluster scaling.



Amazon DynamoDB Annotations



  • @DynamoDBTable


Identifies the target table in Amazon DynamoDB. For example, the following Java code snippet defines a class Developer and maps it to the People table in Amazon DynamoDB.
@DynamoDBTable(tableName="People")
public class Developer { ...}


  • @DynamoDBIgnore


Indicates to the DynamoDBMapper instance that the associated property should be ignored. When saving data to the table, the DynamoDBMapper does not save this property to the table.


  • @DynamoDBAttribute
Maps a property to a table attribute. By default, each class property maps to an item attribute with the same name. However, if the names are not the same, using this tag you can map a property to the attribute. In the following Java snippet, the DynamoDBAttribute maps the BookAuthors property to the Authors attribute name in the table.
@DynamoDBAttribute(attributeName = "Authors")
public List<String> getBookAuthors() { return BookAuthors; }
public void setBookAuthors(List<String> BookAuthors) { this.BookAuthors = BookAuthors; }


The DynamoDBMapper uses Authors as the attribute name when saving the object to the table.


  • @DynamoDBHashKey


Maps a class property to the hash attribute of the table. The property must be one of the supported String or Numeric type and cannot be a collection type.
Assume that you have a table, ProductCatalog, that has Id as the primary key. The following Java code snippet defines a CatalogItem class and maps itsId property to the primary key of the ProductCatalog table using the @DynamoDBHashKey tag.
@DynamoDBTable(tableName="ProductCatalog")
public class CatalogItem {
   private String Id;   
  @DynamoDBHashKey(attributeName="Id")
  public String getId() {
       return Id;
  }
  public void setId(String Id) {
       this.Id = Id;
  }
  // Additional properties go here.
}


  • @DynamoDBRangeKey


Maps a class property to the range key attribute of the table. If the primary key is made of both the hash and range key attributes, you can use this tag to map your class field to the range attribute. For example, assume that you have a Reply table that stores replies for forum threads. Each thread can have many replies. So the primary key of this table is both the ThreadId and ReplyDateTime. The ThreadId is the hash attribute and ReplyDateTime is the range attribute. The following Java code snippet defines a Reply class and maps it to the Reply table. It uses both the @DynamoDBHashKey and @DynamoDBRangeKeytags to identify class properties that map to the primary key.
@DynamoDBTable(tableName="Reply")
public class Reply {
   private String id;
   private String replyDateTime;

   @DynamoDBHashKey(attributeName="Id")
   public String getId() { return id; }
   public void setId(String id) { this.id = id; }

   @DynamoDBRangeKey(attributeName="ReplyDateTime")
   public String getReplyDateTime() { return replyDateTime; }
   public void setReplyDateTime(String replyDateTime) { this.replyDateTime = replyDateTime; }

  // Additional properties go here.
}



  • @DynamoDBAutoGeneratedKey


Marks a hash key or range key property as being auto-generated. The Object Persistence Model will generate a random UUID when saving these attributes. Only String properties can be marked as auto-generated keys.
The following snippet demonstrates using auto-generated keys.
@DynamoDBTable(tableName="AutoGeneratedKeysExample")
public class AutoGeneratedKeys {
   private String id;
   private String payload;
   
   @DynamoDBHashKey(attributeName = "Id")
   @DynamoDBAutoGeneratedKey
   public String getId() { return id; }
   public void setId(String id) { this.id = id; }

   @DynamoDBAttribute(attributeName="payload")
   public String getPayload() { return this.payload };
   public String setPayload(String payload) { this.payload = payload };   
  
   public static void saveItem() {
       AutoGeneratedKeys obj = new AutoGeneratedKeys();
       obj.setPayload("abc123");
       
       // id field is null at this point       
       DynamoDBMapper mapper = new DynamoDBMapper(dynamoDBClient);
       mapper.save(obj);
       
       System.out.println("Object was saved with id " + obj.getId());
   }
}


  • @DynamoDBVersionAttribute


Identifies a class property for storing an optimistic locking version number. DynamoDBMapper assigns a version number to this property when it saves a new item, and increments it each time you update the item. Only number scalar types are supported.


DynamoDBMapper Class




The DynamoDBMapper class is the entry point to Amazon DynamoDB. It provides a connection to Amazon DynamoDB and enables you to access your data in various tables, perform various CRUD operations on items, and execute queries and scans against tables. This class provides the following key operations for you to work with Amazon DynamoDB.


  • save
    • Saves the specified object to the table. The object that you wish to save is the only required parameter for this method. You can provide optional configuration parameters using the DynamoDBMapperConfig object.
    • If an item that has the same primary key does not exist, this method creates a new item in the table. If an item that has the same primary key exists, it updates the existing item. String hash and range keys annotated with @DynamoDBAutoGeneratedKey are given a random universally unique identifier (UUID) if left uninitialized. Version fields annotated with @DynamoDBVersionAttribute will be incremented by one. Additionally, if a version field is updated or a key generated, the object passed in is updated as a result of the operation.


mapper.save(obj, new DynamoDBMapperConfig(DynamoDBMapperConfig.SaveBehavior.CLOBBER));



  • load
    • Retrieves an item from a table. You must provide the primary key of the item that you wish to retrieve.


CatalogItem item = mapper.load(CatalogItem.class, item.getId(),
               new DynamoDBMapperConfig(DynamoDBMapperConfig.ConsistentReads.CONSISTENT));



  • delete
    • Deletes an item from the table. You must pass in an object instance of the mapped class.


  • query
    • Enables the querying of a table. You can query a table only if its primary key is made of both a hash and a range attribute. This method requires you to provide a hash attribute value and a query filter that is applied on the range attribute. A filter expression includes a condition and a value.


String forumName = "Amazon DynamoDB";
String forumSubject = "DynamoDB Thread 1";
String hashKey = forumName + "#" + forumSubject;

long twoWeeksAgoMilli = (new Date()).getTime() - (14L*24L*60L*60L*1000L);
Date twoWeeksAgo = new Date();
twoWeeksAgo.setTime(twoWeeksAgoMilli);
SimpleDateFormat df = new SimpleDateFormat("yyyy-MM-dd'T'HH:mm:ss.SSS'Z'");
String twoWeeksAgoStr = df.format(twoWeeksAgo);


Condition rangeKeyCondition = new Condition()
   .withComparisonOperator(ComparisonOperator.GT.toString())
   .withAttributeValueList(new AttributeValue().withS(twoWeeksAgoStr.toString()));

Reply replyKey = new Reply();
replyKey.setId(hashKey);

DynamoDBQueryExpression<Reply> queryExpression = new DynamoDBQueryExpression<Reply>()
   .withHashKeyValues(replyKey)
   .withRangeKeyCondition("ReplyDateTime", rangeKeyCondition);

List<Reply> latestReplies = mapper.query(Reply.class, queryExpression);


The query returns a collection of Reply objects.


  • scan
    • Scans an entire table. You can specify optional condition filters items based on one or more Condition instances, and you can specify a filter expression for any item attributes.


DynamoDBScanExpression scanExpression = new DynamoDBScanExpression();
       
Map<String, Condition> scanFilter = new HashMap<String, Condition>();
Condition scanCondition = new Condition()
   .withComparisonOperator(ComparisonOperator.EQ.toString())
   .withAttributeValueList(new AttributeValue().withN("0"));

scanFilter.put("Answered", scanCondition);
       
scanExpression.setScanFilter(scanFilter);

List<Thread> unansweredThreads = mapper.scan(Thread.class, scanExpression);


  • The scan method returns a "lazy-loaded" collection. It initially returns only one page of results, and then makes a service call for the next page if needed. To obtain all the matching items, you only need to iterate over the unansweredThreads collection.


  • batchDelete
    • Deletes objects from one or more tables using one or more calls to the AmazonDynamoDB.batchWriteItem method. This method does not provide transaction guarantees.
Book book1 = mapper.load(Book.class, 901);
Book book2 = mapper.load(Book.class, 902);
mapper.batchDelete(Arrays.asList(book1, book2));



  • batchSave
    • Saves objects to one or more tables using one or more calls to the AmazonDynamoDB.batchWriteItem method. This method does not provide transaction guarantees.


Book book1 = new Book();
book1.id = 901;
book1.productCategory = "Book";
book1.title = "Book 901 Title";

Book book2 = new Book();
book2.id = 902;
book2.productCategory = "Book";
book2.title = "Book 902 Title";

mapper.batchSave(Arrays.asList(book1, book2));



  • batchWrite
    • Saves objects to and deletes objects from one or more tables using one or more calls to the AmazonDynamoDB.batchWriteItem method. This method does not provide transaction guarantees or support versioning (conditional puts or deletes).


// Create a Forum item to save
Forum forumItem = new Forum();
forumItem.name = "Test BatchWrite Forum";
       
// Create a Thread item to save
Thread threadItem = new Thread();
threadItem.forumName = "AmazonDynamoDB";
threadItem.subject = "My sample question";
       
// Load a ProductCatalog item to delete
Book book3 = mapper.load(Book.class, 903);
       
List<Object> objectsToWrite = Arrays.asList(forumItem, threadItem);
List<Book> objectsToDelete = Arrays.asList(book3);
      
mapper.batchWrite(objectsToWrite, objectsToDelete);



  • count
    • Evaluates the specified scan expression and returns the count of matching items. No item data is returned.


  • marshallIntoObject
    • Utility method to transform a result from the low-level API into a domain object.



Supported Data Types




Amazon DynamoDB supports the following primitive data types and primitive wrapper classes.
  • String
  • Boolean, boolean
  • Byte, byte
  • Date (as ISO8601 millisecond-precision string, shifted to UTC)
  • Calendar (as ISO8601 millisecond-precision string, shifted to UTC)
  • Long, long
  • Integer, int
  • Double, double
  • Float, float
  • BigDecimal
  • BigInteger
Amazon DynamoDB supports the Java Set collection types. If your mapped collection property is not a Set, then an exception is thrown.
The following table summarizes how the preceding Java types map to the Amazon DynamoDB types.
Java type
Amazon DynamoDB type
All number types
N (number type)
Strings
S (string type)
boolean
N (number type), 0 or 1.
ByteBuffer
B (binary type)
Date
S (string type). The Date values are stored as ISO-8601 formatted strings.
Set collection types
SS (string set) type, NS (number set) type, or BS (binary set) type.



Java Example : CRUD Operations


CatalogItem.java


import java.util.Set;
import com.amazonaws.services.dynamodbv2.datamodeling.DynamoDBAttribute;
import com.amazonaws.services.dynamodbv2.datamodeling.DynamoDBHashKey;
import com.amazonaws.services.dynamodbv2.datamodeling.DynamoDBTable;
@DynamoDBTable(tableName="ProductCatalog")
public class CatalogItem {
    private Integer id;
    private String title;
    private String ISBN;
    private Set bookAuthors;
   
    @DynamoDBHashKey(attributeName="Id")
    public Integer getId() { return id; }
    public void setId(Integer id) { this.id = id; }
   
    @DynamoDBAttribute(attributeName="Title")
    public String getTitle() { return title; }    
    public void setTitle(String title) { this.title = title; }

    @DynamoDBAttribute(attributeName="ISBN")
    public String getISBN() { return ISBN; }    
    public void setISBN(String ISBN) { this.ISBN = ISBN;}
   
    @DynamoDBAttribute(attributeName = "Authors")
    public Set getBookAuthors() { return bookAuthors; }    
    public void setBookAuthors(Set bookAuthors) { this.bookAuthors = bookAuthors; }

    @Override
    public String toString() {
       return "Book [ISBN=" + ISBN + ", bookAuthors=" + bookAuthors
       + ", id=" + id + ", title=" + title + "]";            
    }
}
ObjectPersistenceCRUDExample.java


import java.io.IOException;
import java.util.Arrays;
import java.util.HashMap;
import java.util.HashSet;
import java.util.Map;
import java.util.Random;

import com.amazonaws.AmazonClientException;
import com.amazonaws.AmazonServiceException;
import com.amazonaws.auth.AWSCredentials;
import com.amazonaws.auth.ClasspathPropertiesFileCredentialsProvider;
import com.amazonaws.regions.Region;
import com.amazonaws.regions.Regions;
import com.amazonaws.services.dynamodbv2.AmazonDynamoDBClient;
import com.amazonaws.services.dynamodbv2.datamodeling.DynamoDBMapper;
import com.amazonaws.services.dynamodbv2.datamodeling.DynamoDBMapperConfig;
import com.amazonaws.services.dynamodbv2.model.AttributeDefinition;
import com.amazonaws.services.dynamodbv2.model.AttributeValue;
import com.amazonaws.services.dynamodbv2.model.ComparisonOperator;
import com.amazonaws.services.dynamodbv2.model.Condition;
import com.amazonaws.services.dynamodbv2.model.CreateTableRequest;
import com.amazonaws.services.dynamodbv2.model.DescribeTableRequest;
import com.amazonaws.services.dynamodbv2.model.KeySchemaElement;
import com.amazonaws.services.dynamodbv2.model.KeyType;
import com.amazonaws.services.dynamodbv2.model.ProvisionedThroughput;
import com.amazonaws.services.dynamodbv2.model.ScalarAttributeType;
import com.amazonaws.services.dynamodbv2.model.ScanRequest;
import com.amazonaws.services.dynamodbv2.model.ScanResult;
import com.amazonaws.services.dynamodbv2.model.TableDescription;
import com.amazonaws.services.dynamodbv2.model.TableStatus;

public class ObjectPersistenceCRUDExample {
 static AmazonDynamoDBClient client;
 private DynamoDBMapper mapper;
 private static int PRODUCT_ID;

 public static void main(String[] args) throws IOException {
  ObjectPersistenceCRUDExample demo = new ObjectPersistenceCRUDExample();
  demo.init();
  demo.createTable("ProductCatalog");
  for (int i = 0; i < 100; i++) {
   System.out.println(i);
   PRODUCT_ID = PRODUCT_ID + i;
   demo.insert();
  }
  demo.getAllRows();
  CatalogItem itemRetrieved = demo.load(PRODUCT_ID);
  demo.update(itemRetrieved);
  CatalogItem updatedItem = demo.load(PRODUCT_ID);
  demo.delete(updatedItem);
  demo.load(updatedItem.getId());
  System.out.println("Example complete!");
 }

 private void init() {
  PRODUCT_ID = new Random().nextInt(1000);
  AWSCredentials credentials = new ClasspathPropertiesFileCredentialsProvider()
    .getCredentials();
  client = new AmazonDynamoDBClient(credentials);
  Region usWest2 = Region.getRegion(Regions.US_WEST_2);
  client.setRegion(usWest2);
  mapper = new DynamoDBMapper(client);
 }

 private void createTable(String tableName) {
  try {
   CreateTableRequest createTableRequest = new CreateTableRequest()
     .withTableName(tableName);
   createTableRequest.withKeySchema(new KeySchemaElement()
     .withAttributeName("Id").withKeyType(KeyType.HASH));
   createTableRequest
     .withAttributeDefinitions(new AttributeDefinition()
       .withAttributeName("Id").withAttributeType(
         ScalarAttributeType.N));
   createTableRequest
     .withProvisionedThroughput(new ProvisionedThroughput()
       .withReadCapacityUnits(10L).withWriteCapacityUnits(
         10L));
   TableDescription createdTableDescription = client.createTable(
     createTableRequest).getTableDescription();
   System.out.println("Created Table: " + createdTableDescription);
   // Wait for it to become active
   waitForTableToBecomeAvailable(tableName);
  } catch (AmazonServiceException e) {
   e.printStackTrace();
  } catch (AmazonClientException e) {
   e.printStackTrace();
  }
 }

 private void waitForTableToBecomeAvailable(String tableName) {
  System.out.println("Waiting for " + tableName + " to become ACTIVE...");
  long startTime = System.currentTimeMillis();
  long endTime = startTime + (10 * 60 * 1000);
  while (System.currentTimeMillis() < endTime) {
   try {
    Thread.sleep(1000 * 20);
   } catch (Exception e) {
   }
   try {
    DescribeTableRequest request = new DescribeTableRequest()
      .withTableName(tableName);
    TableDescription tableDescription = client.describeTable(
      request).getTable();
    String tableStatus = tableDescription.getTableStatus();
    System.out.println("  - current state: " + tableStatus);
    if (tableStatus.equals(TableStatus.ACTIVE.toString()))
     return;
   } catch (AmazonServiceException ase) {
    if (ase.getErrorCode().equalsIgnoreCase(
      "ResourceNotFoundException") == false)
     throw ase;
   }
  }
  throw new RuntimeException("Table " + tableName + " never went active");
 }

 private void insert() {
  CatalogItem item = new CatalogItem();
  item.setId(PRODUCT_ID);
  item.setTitle("Book PRODUCT_ID");
  item.setISBN("611-1111111111");
  item.setBookAuthors(new HashSet(Arrays.asList("Author1",
    "Author2")));
  // Save the item (book).
  mapper.save(item);
 }

 private void update(CatalogItem itemRetrieved) {
  itemRetrieved.setISBN("622-2222222222");
  itemRetrieved.setBookAuthors(new HashSet(Arrays.asList(
    "Author1", "Author3")));
  mapper.save(itemRetrieved);
  System.out.println("Item updated:");
  System.out.println(itemRetrieved);
 }

 private void delete(CatalogItem updatedItem) {
  // Delete the item.
  mapper.delete(updatedItem);
 }

 private CatalogItem load(int id) {
  // Retrieve the updated item.
  DynamoDBMapperConfig config = new DynamoDBMapperConfig(
    DynamoDBMapperConfig.ConsistentReads.CONSISTENT);
  CatalogItem updatedItem = mapper.load(CatalogItem.class, id, config);
  if (updatedItem == null) {
   System.out.println("Done - Sample item is deleted.");
  } else {
   System.out.println("Retrieved item:");
   System.out.println(updatedItem);
  }
  return updatedItem;
 }

 private void getAllRows() {
  ScanRequest scanRequest = new ScanRequest()
    .withTableName("ProductCatalog");
  scanRequest.setLimit(10);
  HashMap scanFilter = new HashMap();
  Condition condition = new Condition().withComparisonOperator(
    ComparisonOperator.EQ.toString()).withAttributeValueList(
    new AttributeValue().withS("611-1111111111"));
  scanFilter.put("ISBN", condition);
  Condition condition2 = new Condition().withComparisonOperator(
    ComparisonOperator.LE.toString()).withAttributeValueList(
    new AttributeValue().withN("1000"));
  scanFilter.put("Id", condition2);
  scanRequest.withScanFilter(scanFilter);
  try {
   System.out.println("Scan Request: " + scanRequest);
   ScanResult scanResponse = client.scan(scanRequest);
   for (Map item : scanResponse.getItems()) {
    System.out.println(item.get("Id").getN() + " , " +
    item.get("ISBN").getS() + " , " +
    item.get("Authors").getSS() + " , " +
    item.get("Title").getS());
   }
   System.out.println("Scan Response: " + scanResponse);
   System.out.println("Count: " + scanResponse.getCount());
   System.out.println("Scanned Count: "
     + scanResponse.getScannedCount());
   System.out.println("Items: " + scanResponse.getItems());
  } catch (AmazonServiceException e) {
   e.printStackTrace();
  } catch (AmazonClientException e) {
   e.printStackTrace();
  }
 }
}

Saturday, 26 January 2013

MongoDB is an open source document-oriented database system developed and supported by 10gen. It is part of the NoSQL family of database systems. Instead of storing data in tables as is done in a "classical" relational database, MongoDB stores structured data as JSON-like documents with dynamic schemas (MongoDB calls the format BSON), making the integration of data in certain types of applications easier and faster.

Main Features


  • Ad hoc queries
  • Indexing
  • Replication
  • Load balancing
  • File storage
  • Aggregation (GROUP BY clause)
  • Server-side JavaScript execution
  • Capped collections (Fixed Size Collections)

Architecture




Pros


  • Schema-less: If you have a flexible schema, this is ideal for a document store like MongoDB. This is difficult to implement in a performant manner in RDBMS.
  • Ease of Scale-Out: Scale reads by using replica sets. Scale writes by using sharding (auto balancing). Just fire up another machine and away you go. Adding more machines = adding more RAM over which to distribute your working set.
  • Cost: Depends on which RDBMS of course, but MongoDB is free and can run on Linux, ideal for running on cheaper commodity kit.
  • you can choose what level of consistency you want depending on the value of the data (e.g. faster performance = fire and forget inserts to MongoDB, slower performance = wait till insert has been replicated to multiple nodes before returning).

Cons


  • Data size in MongoDB is typically higher due to e.g. each document has field names stored it
  • Less flexibity with querying (e.g. no JOINs)
  • No support for transactions - certain atomic operations are supported, at a single document level.
  • At the moment Map/Reduce (e.g. to do aggregations/data analysis) is OK, but not intensively fast. So if that's required, something like Hadoop may need to be added into the mix.
  • Less up to date information available/fast evolving product.

Who Uses MongoDB?


  • AppScale
  • Barclays
  • Bitly
  • Business Insider
  • CERN Large Hadron Collider
  • Craigslist
  • Diaspora
  • Disney Interactive Media Group
  • Etsy
  • foursquare
  • Globo.com
  • MTV Networks
  • MOG Technologies
  • Qiqqa
  • Shutterfly
  • SourceForge
  • The Guardian
  • The New York Times
  • Thumbtack
  • Uber
  • Wordnik

Configuring MongoDB on Ubuntu


Step 1) Add 10gen package to source.list
The 10gen package contains the latest mongoDB version, append below line to the end of the file “/etc/apt/sources.list”

deb http://downloads-distro.mongodb.org/repo/ubuntu-upstart dist 10gen

Step 2) Update Package

$  sudo apt-get update

Step 3) Add GPG Key for 10gen Package

$ sudo apt-key adv --keyserver keyserver.ubuntu.com --recv 7F0CEB10

Step 4) Install mongodb-10gen

$ sudo apt-get install mongodb-10gen

Step 5) Verification

$ mongo
MongoDB shell version: 1.8.1
connecting to: test
>


Step6) MongoDB Java Tutorial - http://www.mongodb.org/display/DOCS/Java+Tutorial
Find me on Facebook! Follow me on Twitter!