Since we are running in AWS we need the elasticsearch-cloud-aws plugin to allow for the nodes in the cluster to find each other.
To pull things together we are building a custom docker image based on the official one and simply install the needed plugin. This gives us everything we need to run.
However, to make it all happen there are some caveats.
The official image uses the
/data directory for logs, data and plugins. The image also exposes
/data as a VOLUME. This makes it possible to point the container at a location on the host to keep the heavy write operations for logging and, of course, the data itself out of the container. It also allows for upgrades etc, by simply pointing a container at the data location.
There is a downside to this. The image also places the plugins under
/data/plugins and so when the container starts and sets the volume, the plugins “vanish”. It’s also worth noting that our custom
Dockerfile, which extends the official one seemed to work just fine with this command:
RUN /elasticsearch/bin/plugin install elasticsearch/elasticsearch-cloud-aws/2.4.1
There are no errors generated by this, however the plugin does NOT persist into
/data/plugins! This seems a bit odd, but in the end the
/data location would end up being replaced by the VOLUME regardless.
To work around this our custom
/elasticsearch/plugins, modifies the config for elasticsearch and then installs the plugin:
FROM dockerfile/elasticsearch MAINTAINER Matthias Johnson <firstname.lastname@example.org> # move the ES plugins away from the /data volume where it won't survive ... RUN mkdir /elasticsearch/plugins RUN sed -i 's@plugins:\s*/data/plugins@plugins: /elasticsearch/plugins@' /elasticsearch/config/elasticsearch.yml # install the AWS plugin RUN /elasticsearch/bin/plugin install elasticsearch/elasticsearch-cloud-aws/2.4.1 # Expose ports. # - 9200: HTTP # - 9300: transport EXPOSE 9200 EXPOSE 9300 # start the service ENTRYPOINT [ "/elasticsearch/bin/elasticsearch" ]
Now, we can use the resulting image to spin up the container to run elasticsearch without having to perform the plugin install to the
/data location before starting the container.
This approach should also work nicely for other plugins we may need in the future.