Tag Archives: elasticsearch

filebeat custom index name


filebeat output to elasticsearch indices
filebeat separate index
filebeat log different index

filebeat.inputs:

- type: log
enabled: true
paths:
- /var/log/some/path/*.log
fields:
type: "query"

- type: log
enabled: true
paths:
- /var/log/another.path/*.log
fields:
type: "error"

filebeat.config.modules:
path: ${path.config}/modules.d/*.yml
reload.enabled: false
setup.template.settings:
index.number_of_shards: 3

setup.kibana:
output.elasticsearch:
hosts: ["192.168.1.100:9200"]
index: "newindex-%{[fields.type]:other}-%{+yyyy.MM.dd}"

setup.template.name: "newindex"
setup.template.pattern: "newindex-*"

increase shards elasticsearch – maximum shards open

Moving to ERROR step
org.elasticsearch.common.ValidationException: Validation Failed: 1: this action would add [2] total shards, but this cluster currently has [100]/[100] maximum shards open;
at org.elasticsearch.indices.ShardLimitValidator.validateShardLimit(ShardLimitValidator.java:80) ~[elasticsearch

curl http://localhost:9200/_cat/shards | wc -l
100

curl -XPUT http://localhost:9200/_cluster/settings -H "Content-Type: application/json" -d '{ "persistent": { "cluster.max_shards_per_node": "200" } }'