Add a Server or Two!
| Topics: | Uncategorized |
|---|---|
| Tags: | |
| Share: |
Every week i run into someone that is having performance issues and they are not aware you can just add another server or two or ten. I’ll travel to meet a company and I’ll ask how many servers they are using for Splunk to search/index/report on a terabyte a day. They will say a couple. I’ll then ask how many they have for a similar sized hadoop or data warehouse project. They will say 50 to 100X that number. Look if your going to give these systems 300+ servers, can we please get 15?
Somehow there is a breakdown in our communication that we scale like all other good architectures.
The following are hopefully some easy pictures to help tell the story. It should be extremely simple and straight forward, to the point of being obvious - if not bug me and i’ll try again.
October 28th, 2009 at 12:59 pm
Hey Erik,
In your scaling model are you forwarding the same source data to all indexers or are you splitting the sources up and sending 1/N of the sources to each indexer (where N is your number of indexers)?
October 29th, 2009 at 8:57 am
Hi Dale, sorry for the late reply.
I recommend auto load balancing using splunk forwarders. This model will split the data from (n) forwaders evenly across (m) indexers. Splitting the data evenly has huge search performance benefits. In general there are lots of advantages to using splunk forwarders at the origin and using auto LB.
If you are using syslog its a bit tricker and in that model i might still use a forwarder ( or two ) to listen for syslog but then have that fowrader auto LB across the indexers.
Not sure that helps or if i even answer the right question
If not, let me know and i’ll try again.