Jay Taylor | programmer notes

Jun/16

14

Notes: Installing python grpcio-tools on Mac OS-X (or is it macOS now? ;)

I needed the binary “grpc_python_plugin” to follow the Python gRPC tutorial.

I’ve hit quite a few snags.

And it appears I’m not the only one

pip install grpio-tools

...

grpc/tools/main.cc:33:10: fatal error: 'src/compiler/python_generator.h' file not found

And the grpc docs don’t include macOS instructions.

Let’s start hacking:

wget https://pypi.python.org/packages/7b/22/93b83676787ab07fb7f8d8dcea5351efd6ee62ca0dfba8799cc06f375b37/grpcio_tools-0.14.0.tar.gz#md5=18dd40dd0ffba48bbb8ab865b7fbd23a

tar zxvf grpcio_tools-0.14.0.tar.gz

cd grpcio_tools

I found the python_generator.h file at https://github.com/grpc/grpc/blob/master/src/compiler/python_generator.h, so:

git clone https://github.com/grpc/grpc grpc_root

python setup.py build

...

clang -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -Qunused-arguments -Qunused-arguments -DHAVE_PTHREAD=1 -I. -Igrpc_root -Igrpc_root/include -Ithird_party/protobuf/src -I/usr/local/Cellar/python/2.7.11/Frameworks/Python.framework/Versions/2.7/include/python2.7 -c grpc/tools/main.cc -o build/temp.macosx-10.9-x86_64-2.7/grpc/tools/main.o -frtti -std=c++11
clang -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -Qunused-arguments -Qunused-arguments -DHAVE_PTHREAD=1 -I. -Igrpc_root -Igrpc_root/include -Ithird_party/protobuf/src -I/usr/local/Cellar/python/2.7.11/Frameworks/Python.framework/Versions/2.7/include/python2.7 -c grpc_root/src/compiler/python_generator.cc -o build/temp.macosx-10.9-x86_64-2.7/grpc_root/src/compiler/python_generator.o -frtti -std=c++11
clang -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -Qunused-arguments -Qunused-arguments -DHAVE_PTHREAD=1 -I. -Igrpc_root -Igrpc_root/include -Ithird_party/protobuf/src -I/usr/local/Cellar/python/2.7.11/Frameworks/Python.framework/Versions/2.7/include/python2.7 -c third_party/protobuf/src/google/protobuf/compiler/zip_writer.cc -o build/temp.macosx-10.9-x86_64-2.7/third_party/protobuf/src/google/protobuf/compiler/zip_writer.o -frtti -std=c++11
clang: error: no such file or directory: 'third_party/protobuf/src/google/protobuf/compiler/zip_writer.cc'
clang: error: no input files
error: command 'clang' failed with exit status 1

clang: error: no such file or directory: 'third_party/protobuf/src/google/protobuf/compiler/zip_writer.cc'

Okay, what a mess! Well okay, I found the set of files in question.

mkdir tmp
cd tmp
wget https://android.googlesource.com/platform/external/chromium_org/+archive/00d67fb/third_party/protobuf/src/google/protobuf.tar.gz
tar xzvf protobuf.tar.gz
rm protobuf.tar.gz
mkdir -p ../grpc_root/third_party/protobuf/src/google/protobuf
mv * ../grpc_root/third_party/protobuf/src/google/protobuf
cd ..

Sadly, even after locating the files and lovingly injecting them, it has no effect and the build still errors out with the same error.

Okay, it turns out that was all wrong. The zip_writer.cc is included with the main grpc repository, it just comes from a submodule.

Let’s try just building that:

GRPC_PYTHON_BUILD_WITH_CYTHON=1 pip install .
...
commands.CommandError: could not find grpc_python_plugin (protoc plugin for GRPC Python)

Reviewing the relevant github issue #5378 grpc_python_plugin is not included with pip install grpcio, it became clear that @revantk hit the exact same problem and set of errors.

Here is my final solution:

Just in case it helps someone else..

If you're missing the `grpc_python_plugin` binary on macOS (Mac OS X?):

git clone https://github.com/google/protobuf.git
cd protobuf
./autogen.sh
./configure
make
make install
cd ..

Then:

git clone https://github.com/grpc/grpc
cd grpc
git submodule update --init --recursive
make grpc_python_plugin
cp bins/opt/grpc_python_plugin /usr/local/bin/

After this I was good to go!

RSS Feed

No comments yet.

Leave a comment!

You must be logged in to post a comment.

<<

>>

Find it!