Question or problem about Python programming:
from datetime import datetime
from elasticsearch import Elasticsearch
es = Elasticsearch()
doc = {
'author': 'kimchy',
'text': 'Elasticsearch: cool. bonsai cool.',
'timestamp': datetime(2010, 10, 10, 10, 10, 10)
}
res = es.index(index="test-index", doc_type='tweet', id=1, body=doc)
print(res['created'])
This simples code is returning the following error:
elasticsearch.exceptions.ConnectionTimeout: ConnectionTimeout caused by - ReadTimeoutError(HTTPConnectionPool(host='localhost', port=9200): Read timed out. (read timeout=10))
Very strange, because the server is ready and set (http://localhost:9200/ is returning some json).
How to solve the problem:
Solution 1:
By default, the timeout value is set to 10 secs. If one wants to change the global timeout value, this can be achieved by setting the flag timeout=your-time while creating the object.
If you have already created the object without specifying the timeout value, then you can set the timeout value for particular request by using request_timeout=your-time flag in the query.
es.search(index="my_index",
doc_type="document",
body=get_req_body(),
request_timeout=30)
Solution 2:
The connection timed out problem could occur if you are using Amazon Elastic Search service.
es = Elasticsearch([{'host': 'xxxxxx.us-east-1.es.amazonaws.com', 'port': 443, 'use_ssl': True}])
The above python code where you override the default port from 9200 to 443 and setting the SSL to true will resolve the issue.
If no port is specified, it is trying to connect to the port 9200 in the specified host and fails after time out
Solution 3:
This is nothing to do with increasing your timeout to 30 seconds.
Do people actually think that elastic search should need up to 30 seconds to return one tiny hit?
The way I fixed this problem was go to config/elasticsearch.yml
uncomment the following
http.port: 9200 network.host: 'localhost'
Network.host might be set to 192.168.0.1 which might work But I just changed it to ‘localhost’
Solution 4:
Note that one of the common reasons for timeouts when doing es.search (or es.index) is large query size. For example, in my case of a pretty large ES index size (> 3M documents), doing a search for a query with 30 words took around 2 seconds, while doing a search for a query with 400 words took over 18 seconds. So for a sufficiently large query even timeout=30 won’t save you. An easy solution is to crop the query to the size that can be answered below the timeout.
Increasing timeout or doing retries on timeout will help you if the cause was in traffic, otherwise this might be your culprit.
Solution 5:
Try setting timeout in Elasticsearch initialization:
es = Elasticsearch([{'host': HOST_ADDRESS, 'port': THE_PORT}], timeout=30)
You can even set retry_on_timeout to True and give the max_retries an optional number:
es = Elasticsearch([{'host': HOST_ADDRESS, 'port': THE_PORT}], timeout=30, max_retries=10, retry_on_timeout=True)
Solution 6:
elasticsearch.exceptions.ConnectionTimeout: ConnectionTimeout caused by - ReadTimeoutError(HTTPConnectionPool(host='localhost', port=9200): Read timed out. (read timeout=10))
mean the request didn’t end in the specified time (by default, timeout=10).
This will work with 30 seconds :
res = es.index(index="test-index", doc_type='tweet', id=1, body=doc, timeout=30)
Solution 7:
my personal problem was solved with (timeout = 10000) which was practically never reached because the entries on server were only 7.000 but it had heavy traffic and its resources were being hogged and that was why the connection was dropping
Solution 8:
The reasons for the timeout could be many and it seems worth checking the logs on elasticsearch side (logs/elasticsearch.log) to see the detailed error. In our case, the error on ES was:
primary shard is not active Timeout: [1m]
As described in this post, this was because our disk was full. We had resized it (and the partition) a day ago to take care of that but ES needs to be restarted if the high/low watermark has been hit once (we are on 5.5.x) which we had not done.
Simply restarting the ES on production resolved the issue for us.
Solution 9:
Two options that help:
1: increase the timeout
Setting a timeout solved this problem for me. Note that newer versions need a unit, e.g. timeout="60s":
es.index(index=index_name, doc_type="domains", id=domain.id, body=body, timeout="60s")
Without a unit, for example by setting timeout=60, you’ll get
elasticsearch.exceptions.RequestError: RequestError(400, 'illegal_argument_exception', 'failed to parse setting [timeout] with value [60] as a time value: unit is missing or unrecognized')
2: reduce text length
It also helps to reduce the text length, e.g. by cutting of long texts, so elastic can store the text faster which will avoid timeouts, too:
es.index(index=index_name, doc_type="domains", id=domain.id, body=text[:5000], timeout="60s")
