Create an account

Very important

  • To access the important data of the forums, you must be active in each forum and especially in the leaks and database leaks section, send data and after sending the data and activity, data and important content will be opened and visible for you.
  • You will only see chat messages from people who are at or below your level.
  • More than 500,000 database leaks and millions of account leaks are waiting for you, so access and view with more activity.
  • Many important data are inactive and inaccessible for you, so open them with activity. (This will be done automatically)


Thread Rating:
  • 476 Vote(s) - 3.44 Average
  • 1
  • 2
  • 3
  • 4
  • 5
Cassandra Schema Management with CQL scripts

#1
To Cassandra Experts: I am tasked with a work of coming up with recommendations on Cassandra CQL script management and deployment. How teams manage (should manage) large number of CQL scripts (schema definition scripts (DDL), data manipulation scripts (INSERT/UPDATE/DELETE) from the inception of Cassandra development and through subsequent changes to the application schema model. If I may, I would like to point out that the development team size is not that small (10+ developers per application functionality area).

One way (probably the wrong way) is to do what a typical relational database shop would do: app developers or development dbas design and create ddl, dml et., scripts, store and maintain them in version control system (e.g. SVN), and deploy the scripts in an environment (dev, qc, etc. ) using some automation (may be as simple as shell or perl script). I think where this breaks down in NoSQL solution such as Cassandra is the actors involved in these three steps.
1 - design and create CQL scripts - should this be done by DevOps (cassandra admins) or application developers?
(2) store and maintain them in SVN - should this be done similar to (1) above and (3) deployment of scripts - should someone from application development do this (or) DevOps do this?
I would also like to get answers from application schema control and auditing viewpoint. For example, for #1 and #2 above, if application developers design, create and store the CQL scripts in SVN, how can one be able to control what gets into the CQL schema and prevent costly errors. If there is dedicated, single team owning the data model rather than all cassandra developers (akin to DBA/Administrators), it is easier to achieve that control.

I am hoping someone those who have done this before could shed some insight into the choices and best practices for CQL code development, deployment and maintenance in a large environment.
Thanks as always.
Reply

#2
I think the main issue you'll be confronted with is that you'll need to write code to perform some migrations, which is a significant difference to applying delta patches in a typical SQL scenario. Basic changes to the schema (as defined using CQL) can easily be applied using the `cqlsh` tool in a DevOps/DBA style. These types of changes would include adding columns and removing columns. But if you need to do something more fundamental, then you're going to have to write CQL client code to migrate the old data. This is especially true the more denormalization and non-declarative indexing your app requires.

FWIW and YMMV I was able to automate one aspect of CQL schema management which was to find a way to keep the schema and the application code in sync. To achieve this I wrote a [CQL schema compiler][1] that generates boilerplate application source code so that data binding is always in sync with the current schema in Cassandra. But that is just one aspect of the overall problem.

[1]:

[To see links please register here]

Reply



Forum Jump:


Users browsing this thread:
1 Guest(s)

©0Day  2016 - 2023 | All Rights Reserved.  Made with    for the community. Connected through