##Patent Idea, Multi-Channel Parallel Synchronous/Asynchronous replication

In distributed systems, everyone is familiar with synchronous replication and asynchronous replication process. When payload is written to a particular node, replication gets kicked in either in a sequential manner (synchronously) replicating one node after another node or in a silent (asynchronous) manner replicating bits and pieces on one or multiple nodes.

My idea is to propose an intermediate "replication manager" which manages/negotiates with the associated replicating nodes. The manager performs synchronous / asynchronous replication by opening multi-channel port connectivity to the replicating node to move the data in parallel multi-threaded faster mode depending on the resources availability. This should yield in faster replication than the traditional one connection replication. If the resources are busy, this mechanism falls into one connection mode.

MySQL and NoSQL at Craigslist

Came across this video on the composite database implementation, some of the challenges faced, architecture at Craigslist.

Cassandra Summit 2012 :: At your tips now

If you have missed the Cassandra Summit 2012, please go through this video playlist. It is amazing to see the cool new features coming up in Cassandra next release while knowing the hickups faced and experiences shared by some of the contributors of the summit.

Hadoop Summit 2012 :: Sit back, relax and watch it

If you have missed the Hadoop Summit 2012, you can sit back relax and watch it at your own leisure now. The summit recordings are now uploaded and available. I would highly recommend everyone to watch these videos without failure. It is the mother of big data these days.

AWS NY Summit 2012

Watch below the AWS NY Summit 2012 talking about deploying enterprise apps in the cloud, best practises, building scalable database, disaster recovery, optimizing costs with AWS, acclerating oraganizations profits with cloud deployments, cloud vs datacenter cost comparisons, building applications with AWS etc.

Strata 2012 :: Data Conference, Orielly

Its amazing to watch the quality of the research and vision these conferences provide. The finest way to invent is to learn from each other and think beyond boundaries. Watch these at your leisure and enlighten yourself with the knowledge of data.

SQL to No-SQL migration :: Baby Step 1 -- Preparing your new table structure

No-SQL has been an evolving norm these days. With plethora of data growing and multiplying day-by-day at an exponential rate, everyone is faced with a challenge to live-up to the new evolving standards for applications, grid computation, analytics, high-availability, fast responding user interfaces, etc.

One of the primary steps in migrating from the traditional RDBMS system to NO-SQL system is to come-up with new table design. Ironically, there is no direct mechanism that converts traditional RDBMS database schema to a NO-SQL table structure. 

While RDBMS based systems evolve from a centralized database design to user interface development, newer No-SQL systems need to be designed based on the developing user interface and the queries needed for connecting the user interface together screen-by-screen one at a time.

There is however a small correlation on how the RDBMS table data can be represented in No-SQL table structure based on the mappings.

Example 1 : A one-to-one mapping of our traditional RDBMS table design will result in the following No-SQL design.

*****RDBMS Design****

Customer
----------------
Customer_id int,
first_name varchar,
last_name varchar,
email_id varchar
CustomerAddress
----------------------
Customer_id int,
Street_addr varchar,
apt varchar,
city varchar,
state varchar,
country varchar
pin varchar

******No-SQL Design******
Eliminate two tables and merge the columns into one table per customer_id

Customer
-----------
Customer_id int,
first_name varchar,
last_name varchar,
email_id varchar,
Street_addr varchar,
apt varchar,
......


Example 2 : A many-to-one and one-to-many mapping of our traditional RDBMS table design will result in the following No-SQL design

****RDBMS Design*****
--------------------------

Customer
---------------
Customer_id int,
first_name varchar,
last_name varchar,
....

CustomerPhone
--------------------
Customer_id int,
Phone_type varchar, //mobile, home, office
Phone_number varchar


****** No-SQL Design *****
-------------------------------

CustomerByPhoneNumber
------------------------
Customer_id int,
Phone_type varchar,
Phone_number varchar,
PRIMARY KEY (Phone_number)


PhoneByCustomerId
---------------------------
Customer_id int,
Phone_type varchar,
Phone_number varchar,
PRIMARY KEY (Customer_id)
  

Example 3 : Many-to-Many
--------------------------------

Many-to-Many in NoSQL is nothing but iteration of one-to-many and many-to-one queries and consolidating the result set.

In the above example, if you want to get list of all customers and all their phone numbers, you would have to perform two queries and iterate over to get complete set.