Answering my own question, showing all the steps I have done.
Upgrading volume for hot/warm backups from 250 IOPS to 1200 IOPS, didn't sort the memore usage patterns. But anyway high iops volumes are good things
On Search heads and indexers I had unix app that produced some errors, but it didn't make any problems before so I didn't look at it at that time. When I removed unix app ( and other defaults one ) it helped a bit, but after couple of minutes memory was starting to go up.
upgraded from 4.3.6 to 5.0.3. The process was straightforward, dpkg -i the package. No more memory leaks. Memory stays at 2-4%.
... View more