NSQ

·

Collect NSQ operation data and report it to TrueWatch in the form of indicators.

Configuration¶

Preconditions¶

NSQ installed（NSQ official website）
Recommend NSQ version >= 1.0.0, already tested version:
1.2.1
1.1.0
0.3.8

Collector Configuration¶

Host InstallationKubernetes

Go to the conf.d/nsq directory under the DataKit installation directory, copy nsq.conf.sample and name it nsq.conf. Examples are as follows:

[[inputs.nsq]]
  ## NSQ Lookupd HTTP API endpoint
  lookupd = "http://localhost:4161"

  ## NSQD HTTP API endpoint
  ## example:
  ##   ["http://localhost:4151"]
  nsqd = []

  ## time units are "ms", "s", "m", "h"
  interval = "10s"

  ## Set true to enable election
  election = true

  ## Optional TLS Config
  # tls_ca = "/etc/telegraf/ca.pem"
  # tls_cert = "/etc/telegraf/cert.pem"
  # tls_key = "/etc/telegraf/key.pem"
  ## Use TLS but skip chain & host verification
  # insecure_skip_verify = false

  [inputs.nsq.tags]
  # some_tag = "some_value"
  # more_tag = "some_other_value"

Once configured, restart DataKit.

The collector can now be turned on by ConfigMap Injection Collector Configuration.

Tow mode for NSQ collector

The NSQ collector is available in two configurations, lookupd and nsqd, as follows:

lookupd: Configure the lookupd address of the NSQ cluster, and the collector will automatically discover the NSQ Server and collect data, which is more scalable.
nsqd: Configure a fixed list of NSQ Daemon (nsqd) addresses for which the collector collects only NSQ Server data

The above two configuration methods are mutually exclusive, and lookupd has higher priority, so it is recommended to use lookupd configuration method.

Metric¶

For all of the following data collections, the global election tags will added automatically, we can add extra tags in [inputs.nsq.tags] if needed:

 [inputs.nsq.tags]
  # some_tag = "some_value"
  # more_tag = "some_other_value"
  # ...

`nsq_topics`¶

Metrics of all topics in the NSQ cluster

Tags & Fields	Description
channel (`tag`)	Channel name
topic (`tag`)	Topic name
backend_depth	Total number of unconsumed messages exceeding the max-queue-size. Type: int Unit: count
deferred_count	Number of messages that have been requeued and are not yet ready for re-sending. Type: int Unit: count
depth	Total number of unconsumed messages in the current channel. Type: int Unit: count
in_flight_count	Number of messages during the sending process or client processing that have not been sent FIN, REQ (requeued), or timed out. Type: int Unit: count
message_count	Total number of messages processed in the current channel. Type: int Unit: count
requeue_count	Number of messages that have timed out or have been sent REQ by the client. Type: int Unit: count
timeout_count	Number of messages that have timed out and are still unprocessed. Type: int Unit: count

`nsq_nodes`¶

Metrics of all nodes in the NSQ cluster.

Tags & Fields	Description
host (`tag`)	Hostname
server_host (`tag`)	Service address, that is `host:ip`.
backend_depth	Total number of unconsumed messages exceeding the max-queue-size. Type: int Unit: count
depth	Total number of unconsumed messages in the current node. Type: int Unit: count
message_count	Total number of messages processed by the current node. Type: int Unit: count

`collector`¶

Tags & Fields	Description
instance (`tag`)	Server addr of the instance
job (`tag`)	Server name of the instance
up	Type: int Unit: -

Custom Object¶

`mq`¶

Tags & Fields	Description
col_co_status (`tag`)	Current status of collector on instance(`OK/NotOK`)
host (`tag`)	The server host address
ip (`tag`)	Connection IP of the instance
name (`tag`)	Object uniq ID
reason (`tag`)	If status not ok, we'll get some reasons about the status
display_name	Displayed name in UI Type: string Unit: N/A
uptime	Current instance uptime Type: int Unit: time,s
version	Current version of the instance Type: string Unit: N/A

NSQ