Jay Taylor's notes
back to listing indexsharding - Google Groups
[web search]
Original source (groups.google.com)
Tags:
groups.google.com
Clipped on: 2013-02-01
sharding 7 posts by 2 authors in Squeryl | Here's a scheme that would be more strongly typed : trait ShardingFunction[K] { def selectShard(k: K): Session } class ShardedTable[A,K](table: Table[A], invokeShardedFieldGetter: A => AnyRef, shardedFieldMetaData: FieldMetaData, f: ShardingFunction[K]) { def shardedLookup(k: K)(implicit dsl: QueryDsl) = {
import dsl._ using(f.selectShard(k)) { val q = from(table)(a => dsl.where { FieldReferenceLinker.createEqualityExpressionWithLastAccessedFieldReferenceAndConstant(invokeShardedFieldGetter(a), k)
} select(a)) q } } } class ShardedTablePrecursor[A](table: Table[A]) { def shardedOn[K,T](f: A => T)(implicit dsl: QueryDsl, ev: T => TypedExpressionNode[K], sf: ShardingFunction[K]) = {
import dsl._ val q = from(table)(a=> select(a)) : Query[A] val n = Utils.mapSampleObject(q, (a0:A) => ev(f(a0)))
val invoker = f.asInstanceOf[A=>AnyRef] new ShardedTable[A,K](table, invoker, n._fieldMetaData, sf) } } object YourSchema extends Schema { implicit object yourHashFunction extends ShardingFunction[Long] { def selectShard(k: Long): Session = sys.error("implemente me !")
} Then you define a sharded table : val shardedTable = table[YourObject].shardedOn(_.theShardingKey) shardedTable.shardedLookup(k) // here it is enforced at compile time that k is the same type as _.theShardingKey ...
Now the insert and updates are left as an exercise ! ;-) It probably would be wise to disable the partial update function, i.e. only update with def update(a: A),
or maybe provide a partial update that takes a shard key in a first argument list 2012/3/14 Artie Pesh-Imam <ar...@cx.com> - show quoted text - Seuls les poissons morts nagent avec le courant
The ShardedTable class could also have this method : def shardedTransaction[U](shardKey: K)(block: Table[A] => U)(implicit dsl: QueryDsl) = dsl.transaction(f.selectShard(shardKey)) {
block(table) } shardedTable.shardedTransaction(aValueToShardOn) { table => // the 'raw' table is available here // squeryl code here will run against the chosen shard,
table.insert(...) // inserts, updates and deletes will magically be ran against the correct shard... } 2012/3/14 Maxime Lévesque <maxime....@gmail.com> - show quoted text - - show quoted text -
Yes... there needs to be this implicit conversion in the the Schema trait : protected implicit def tableToSharded[A](t: Table[A]) = new ShardedTablePrecursor(t)
2012/3/15 Artie Pesh-Imam <ar...@cx.com> - show quoted text - - show quoted text -
The more self contained and clean, the more I'll want to pull it ;-) I will change to an sbt multiproject setup pretty soon, sharding could go in it's own sub project. Contact me privately, we can discuss it.
BTW, did you read this : http://www.codefutures.com/database-sharding/ ? Powered by Evernote Publisher
|