Jay Taylor's notes

back to listing index

Riak - all partitions belong to one node

[web search]
Original source (stackoverflow.com)
Tags: riak distributed-systems troubleshooting stackoverflow.com
Clipped on: 2016-05-31

I'm going through the book Seven Databases in Seven Weeks (a good read so far), and I'm confused on a Riak detail that is passed over quickly in the book.

Riak is supposed to, by default, break the data up into 64 partitions. Each of these partitions are supposed to be split up between the nodes in the ring. (Correct me if I got the lingo wrong.)

I am using 4 dev nodes that come with the Riak source. All of them are started, but when I curl http://localhost:8091/stats | grep ring, I see

"ring_ownership": "[{'dev1@127.0.0.1',64}]"

This is further confirmed with a $RIAK_INSTALL/dev/dev4/bin/riak-admin member-status:

================================= Membership ==================================
Status     Ring    Pending    Node
-------------------------------------------------------------------------------
joining     0.0%      --      'dev2@127.0.0.1'
joining     0.0%      --      'dev3@127.0.0.1'
joining     0.0%      --      'dev4@127.0.0.1'
valid     100.0%      --      'dev1@127.0.0.1'
-------------------------------------------------------------------------------

What's going on? Why has the dev1 node claimed all of the partitions, and how can I make it share?

Perhaps related

I edited app.config for each node (in RIAK_ROOT/dev/devN/etc/app.config) to make the pb_ip 0.0.0.0 instead of 127.0.0.1. This was so that I can access Riak from a browser in my host machine, even though I'm running Riak in a Vagrant VM. Even though I made this same change in each of them, I can only access dev1 from my host's browser (not dev2, dev3, or dev4).

If you think it would help, I can package up this VM and make it available to help you help me troubleshoot. (One of the many reasons VMs are awesome.)

asked Nov 26 '12 at 23:39
Image (Asset 2/4) alt=
chadoh
1,97021842
up vote 4 down vote accepted

I suspect you're seeing that output riak-admin member-status because you have outstanding changes to your cluster which need to be committed.

riak-admin cluster plan

riak-admin cluster commit

Running riak-admin cluster plan will show information on transfers that are outstanding, if any. You then need to commit the changes to your cluster with the second command.

answered Nov 27 '12 at 2:20
Image (Asset 3/4) alt=
mafrosis
1,24311219
   upvote
  flag
Nice! I'm curious as to how I ended up in that state, but I'm happy just getting past it for now. Each node now has 16 partitions. Any idea why only dev1 responds in my host browser? – chadoh Nov 27 '12 at 14:32
1 upvote
  flag
Each dev node is listening on different ports. That's how a devrel cluster is setup by the build scripts. You're running in a test environment. In a deployed environment each node (read: separate system or virtual machine) will bind to the same port, the one that dev1 listens on, and you can load balance between them. – Greg Burd Nov 27 '12 at 17:48
   upvote
  flag
@GregoryBurd I get that, but it seems to me that I ought to be able to hit localhost:8092 to directly access dev2, localhost:8093 to access dev3, etc. I can curl those URLs in my (headless) VM, but I can only access dev1 (at localhost:8091) from my host machine. Not a big deal, but I'm not sure why. – chadoh Nov 27 '12 at 18:48

The dangers of publishing works on high velocity projects is that the interface changes before the ink dries.

There was a major shift in cluster management between 1.0 (what the book covers) and 1.2 (the current version). The book just issued a join directly to dev1 and called it a day. Now you must go through the riak-admin cluster command, which batches multiple joins/leaves and executes them as a single transaction. Once you've joined, you must view the plan and commit the transaction, as mentioned in another comment.

FWIW, most of the remaining Riak information is still the same.

answered Nov 27 '12 at 17:53
Image (Asset 4/4) alt=
Coderoshi
34335

Your Answer

asked

3 years ago

viewed

685 times

active

3 years ago

Hot Network Questions

Technology Life / Arts Culture / Recreation Science Other
  1. Stack Overflow
  2. Server Fault
  3. Super User
  4. Web Applications
  5. Ask Ubuntu
  6. Webmasters
  7. Game Development
  8. TeX - LaTeX
  1. Programmers
  2. Unix & Linux
  3. Ask Different (Apple)
  4. WordPress Development
  5. Geographic Information Systems
  6. Electrical Engineering
  7. Android Enthusiasts
  8. Information Security
  1. Database Administrators
  2. Drupal Answers
  3. SharePoint
  4. User Experience
  5. Mathematica
  6. Salesforce
  7. ExpressionEngine® Answers
  8. more (13)
  1. Photography
  2. Science Fiction & Fantasy
  3. Graphic Design
  4. Movies & TV
  5. Seasoned Advice (cooking)
  6. Home Improvement
  7. Personal Finance & Money
  8. Academia
  9. more (9)
  1. English Language & Usage
  2. Skeptics
  3. Mi Yodeya (Judaism)
  4. Travel
  5. Christianity
  6. Arqade (gaming)
  7. Bicycles
  8. Role-playing Games
  9. more (21)
  1. Mathematics
  2. Cross Validated (stats)
  3. Theoretical Computer Science
  4. Physics
  5. MathOverflow
  6. Chemistry
  7. Biology
  8. more (5)
  1. Stack Apps
  2. Meta Stack Exchange
  3. Area 51
  4. Stack Overflow Careers
site design / logo © 2016 Stack Exchange Inc; user contributions licensed under cc by-sa 3.0 with attribution required
rev 2016.5.30.3623