Skip to content

Solr

·

Solr collector, which collects statistics of Solr Cache, Request Times, and so on.

Configuration

Preconditions

DataKit uses the Solr Metrics API to collect metrics data and supports Solr 7.0 and above. Available for Solr 6.6, but the indicator data is incomplete.

Already tested version:

  • 8.11.2
  • 7.0.0

Collector Configuration

Go to the conf.d/db directory under the DataKit installation directory, copy solr.conf.sample and name it solr.conf. Examples are as follows:

[[inputs.solr]]
  ##(optional) collect interval, default is 10 seconds
  interval = '10s'

  ## specify a list of one or more Solr servers
  servers = ["http://localhost:8983"]

  ## Optional HTTP Basic Auth Credentials
  # username = "username"
  # password = "pa$$word"

  ## Set true to enable election
  election = true

  # [inputs.solr.log]
  # files = []
  # #grok pipeline script path
  # pipeline = "solr.p"

  [inputs.solr.tags]
  # some_tag = "some_value"
  # more_tag = "some_other_value"

After configuration, restart DataKit.

The collector can now be turned on by ConfigMap Injection Collector Configuration.

To collect Solr's log, open files in Solr.conf and write to the absolute path of the Solr log file. For example:

[inputs.solr.log]
    # fill in the absolute path
    files = ["/path/to/demo.log"]

Metric

For all of the following data collections, the global election tags will added automatically, we can add extra tags in [inputs.solr.tags] if needed:

 [inputs.solr.tags]
  # some_tag = "some_value"
  # more_tag = "some_other_value"
  # ...

solr_cache

Tags & Fields Description
category
(tag)		 Category name.
core
(tag)		 Solr core.
group
(tag)		 Metric group.
host
(tag)		 System hostname.
instance
(tag)		 Instance name, generated based on server address.
name
(tag)		 Cache name.
cumulative_evictions Number of cache evictions across all caches since this node has been running.
Type: int
Unit: count
cumulative_hitratio Ratio of cache hits to lookups across all the caches since this node has been running.
Type: float
Unit: percent,percent
cumulative_hits Number of cache hits across all the caches since this node has been running.
Type: int
Unit: count
cumulative_inserts Number of cache insertions across all the caches since this node has been running.
Type: int
Unit: count
cumulative_lookups Number of cache lookups across all the caches since this node has been running.
Type: int
Unit: count
evictions Number of cache evictions for the current index searcher.
Type: int
Unit: count
hitratio Ratio of cache hits to lookups for the current index searcher.
Type: float
Unit: percent,percent
hits Number of hits for the current index searcher.
Type: int
Unit: count
inserts Number of inserts into the cache.
Type: int
Unit: count
lookups Number of lookups against the cache.
Type: int
Unit: count
max_ram Maximum heap that should be used by the cache beyond which keys will be evicted.
Type: int
Unit: digital,MB
ram_bytes_used Actual heap usage of the cache at that particular instance.
Type: int
Unit: digital,B
size Number of entries in the cache at that particular instance.
Type: int
Unit: count
warmup Warm-up time for the registered index searcher. This time is taken in account for the "auto-warming" of caches.
Type: int
Unit: time,ms

solr_request_times

Tags & Fields Description
category
(tag)		 Category name.
core
(tag)		 Solr core.
group
(tag)		 Metric group.
handler
(tag)		 Request handler.
host
(tag)		 System hostname.
instance
(tag)		 Instance name, generated based on server address.
count Total number of requests made since the Solr process was started.
Type: int
Unit: count
max Max of all the request processing time.
Type: float
Unit: time,ms
mean Mean of all the request processing time.
Type: float
Unit: time,ms
median Median of all the request processing time.
Type: float
Unit: time,ms
min Min of all the request processing time.
Type: float
Unit: time,ms
p75 Request processing time for the request which belongs to the 75th Percentile.
Type: float
Unit: time,ms
p95 Request processing time in milliseconds for the request which belongs to the 95th Percentile.
Type: float
Unit: time,ms
p99 Request processing time in milliseconds for the request which belongs to the 99th Percentile.
Type: float
Unit: time,ms
p999 Request processing time in milliseconds for the request which belongs to the 99.9th Percentile.
Type: float
Unit: time,ms
rate_15min Requests per second received over the past 15 minutes.
Type: float
Unit: throughput,reqps
rate_1min Requests per second received over the past 1 minutes.
Type: float
Unit: throughput,reqps
rate_5min Requests per second received over the past 5 minutes.
Type: float
Unit: throughput,reqps
rate_mean Average number of requests per second received
Type: float
Unit: throughput,reqps
stddev Stddev of all the request processing time.
Type: float
Unit: time,ms

solr_searcher

Tags & Fields Description
category
(tag)		 Category name.
core
(tag)		 Solr core.
group
(tag)		 Metric group.
host
(tag)		 System hostname.
instance
(tag)		 Instance name, generated based on server address.
deleted_docs The number of deleted documents.
Type: int
Unit: count
max_docs The largest possible document number.
Type: int
Unit: count
num_docs The total number of indexed documents.
Type: int
Unit: count
warmup The time spent warming up.
Type: int
Unit: time,ms

Log Collection

Example of cutting logs:

2013-10-01 12:33:08.319 INFO (org.apache.solr.core.SolrCore) [collection1] webapp.reporter

Cut fields:

Field Name Field Value
Reporter webapp.reporter
status INFO
thread org.apache.solr.core.SolrCore
time 1380630788319000000