Jay Taylor's notes

back to listing index

kafka 8 and memory - There is insufficient memory for the Java Runtime Environment to continue

[web search]
Original source (stackoverflow.com)
Tags: java kafka stackoverflow.com
Clipped on: 2015-10-20

I am using DigiOcean instance with 512 megs of ram, I get the below error with kafka. I am not a java proficient dev. How do I adjust kafka to utilize the small amount of ram. This is a dev sever. I dont want to pay more per hour for a bigger machine.

#
# There is insufficient memory for the Java Runtime Environment to continue.
# Native memory allocation (malloc) failed to allocate 986513408 bytes for committing reserved memory.
# An error report file with more information is saved as:
# //hs_err_pid6500.log
OpenJDK 64-Bit Server VM warning: INFO: os::commit_memory(0x00000000bad30000, 986513408, 0) failed; error='Cannot allocate memory' (errno=12)
asked Jan 30 '14 at 5:55
Image (Asset 2/4) alt=
Tampa
7,47630100187
up vote 15 down vote accepted

You can adjust the JVM heap size by editing kafka-server-start.sh, zookeeper-server-start.shand so on:

export KAFKA_HEAP_OPTS="-Xmx1G -Xms1G"

The -Xms parameter specifies the minimum heap size. To get your server to at least start up, try changing it to use less memory. Given that you only have 512M, you should change the maximum heap size (-Xmx) too:

export KAFKA_HEAP_OPTS="-Xmx256M -Xms128M"

I'm not sure what the minimal memory requirements of kafka in default config are - maybe you need to adjust the message size in kafka to get it to run.

answered Jan 30 '14 at 7:27
Image (Asset 3/4) alt=
Chris
2,4431221

Area: HotSpot / gc

Synopsis

Crashes due to failure to allocate large pages.

On Linux, failures when allocating large pages can lead to crashes. When running JDK 7u51 or later versions, the issue can be recognized in two ways:

    Before the crash happens, one or more lines similar to the following example will have been printed to the log:

    os::commit_memory(0x00000006b1600000, 352321536, 2097152, 0) failed;
    error='Cannot allocate memory' (errno=12); Cannot allocate large pages, 
    falling back to regular pages

    If a file named hs_err is generated, it will contain a line similar to the following example:

    Large page allocation failures have occurred 3 times

The problem can be avoided by running with large page support turned off, for example, by passing the "-XX:-UseLargePages" option to the java binary.
answered May 11 at 11:17
Image (Asset 4/4) alt=

Your Answer

asked

1 year ago

viewed

4279 times

active

5 months ago
Technology Life / Arts Culture / Recreation Science Other
  1. Stack Overflow
  2. Server Fault
  3. Super User
  4. Web Applications
  5. Ask Ubuntu
  6. Webmasters
  7. Game Development
  8. TeX - LaTeX
  1. Programmers
  2. Unix & Linux
  3. Ask Different (Apple)
  4. WordPress Development
  5. Geographic Information Systems
  6. Electrical Engineering
  7. Android Enthusiasts
  8. Information Security
  1. Database Administrators
  2. Drupal Answers
  3. SharePoint
  4. User Experience
  5. Mathematica
  6. Salesforce
  7. ExpressionEngine® Answers
  8. more (13)
  1. Photography
  2. Science Fiction & Fantasy
  3. Graphic Design
  4. Movies & TV
  5. Seasoned Advice (cooking)
  6. Home Improvement
  7. Personal Finance & Money
  8. Academia
  9. more (9)
  1. English Language & Usage
  2. Skeptics
  3. Mi Yodeya (Judaism)
  4. Travel
  5. Christianity
  6. Arqade (gaming)
  7. Bicycles
  8. Role-playing Games
  9. more (21)
  1. Mathematics
  2. Cross Validated (stats)
  3. Theoretical Computer Science
  4. Physics
  5. MathOverflow
  6. Chemistry
  7. Biology
  8. more (5)
  1. Stack Apps
  2. Meta Stack Exchange
  3. Area 51
  4. Stack Overflow Careers
site design / logo © 2015 Stack Exchange Inc; user contributions licensed under cc by-sa 3.0 with attribution required
rev 2015.10.20.2890