1 .. SPDX-License-Identifier: BSD-3-Clause
3 =======================
4 Introduction to Netlink
5 =======================
7 Netlink is often described as an ioctl() replacement.
8 It aims to replace fixed-format C structures as supplied
9 to ioctl() with a format which allows an easy way to add
10 or extended the arguments.
12 To achieve this Netlink uses a minimal fixed-format metadata header
13 followed by multiple attributes in the TLV (type, length, value) format.
15 Unfortunately the protocol has evolved over the years, in an organic
16 and undocumented fashion, making it hard to coherently explain.
17 To make the most practical sense this document starts by describing
18 netlink as it is used today and dives into more "historical" uses
24 Netlink communication happens over sockets, a socket needs to be
29 fd = socket(AF_NETLINK, SOCK_RAW, NETLINK_GENERIC);
31 The use of sockets allows for a natural way of exchanging information
32 in both directions (to and from the kernel). The operations are still
33 performed synchronously when applications send() the request but
34 a separate recv() system call is needed to read the reply.
36 A very simplified flow of a Netlink "call" will therefore look
41 fd = socket(AF_NETLINK, SOCK_RAW, NETLINK_GENERIC);
43 /* format the request */
44 send(fd, &request, sizeof(request));
45 n = recv(fd, &response, RSP_BUFFER_SIZE);
46 /* interpret the response */
48 Netlink also provides natural support for "dumping", i.e. communicating
49 to user space all objects of a certain type (e.g. dumping all network
54 fd = socket(AF_NETLINK, SOCK_RAW, NETLINK_GENERIC);
56 /* format the dump request */
57 send(fd, &request, sizeof(request));
59 n = recv(fd, &buffer, RSP_BUFFER_SIZE);
60 /* one recv() call can read multiple messages, hence the loop below */
61 for (nl_msg in buffer) {
62 if (nl_msg.nlmsg_type == NLMSG_DONE)
64 /* process the object */
69 The first two arguments of the socket() call require little explanation -
70 it is opening a Netlink socket, with all headers provided by the user
71 (hence NETLINK, RAW). The last argument is the protocol within Netlink.
72 This field used to identify the subsystem with which the socket will
75 Classic vs Generic Netlink
76 --------------------------
78 Initial implementation of Netlink depended on a static allocation
79 of IDs to subsystems and provided little supporting infrastructure.
80 Let us refer to those protocols collectively as **Classic Netlink**.
81 The list of them is defined on top of the ``include/uapi/linux/netlink.h``
82 file, they include among others - general networking (NETLINK_ROUTE),
83 iSCSI (NETLINK_ISCSI), and audit (NETLINK_AUDIT).
85 **Generic Netlink** (introduced in 2005) allows for dynamic registration of
86 subsystems (and subsystem ID allocation), introspection and simplifies
87 implementing the kernel side of the interface.
89 The following section describes how to use Generic Netlink, as the
90 number of subsystems using Generic Netlink outnumbers the older
91 protocols by an order of magnitude. There are also no plans for adding
92 more Classic Netlink protocols to the kernel.
93 Basic information on how communicating with core networking parts of
94 the Linux kernel (or another of the 20 subsystems using Classic
95 Netlink) differs from Generic Netlink is provided later in this document.
100 In addition to the Netlink fixed metadata header each Netlink protocol
101 defines its own fixed metadata header. (Similarly to how network
102 headers stack - Ethernet > IP > TCP we have Netlink > Generic N. > Family.)
104 A Netlink message always starts with struct nlmsghdr, which is followed
105 by a protocol-specific header. In case of Generic Netlink the protocol
106 header is struct genlmsghdr.
108 The practical meaning of the fields in case of Generic Netlink is as follows:
113 __u32 nlmsg_len; /* Length of message including headers */
114 __u16 nlmsg_type; /* Generic Netlink Family (subsystem) ID */
115 __u16 nlmsg_flags; /* Flags - request or dump */
116 __u32 nlmsg_seq; /* Sequence number */
117 __u32 nlmsg_pid; /* Port ID, set to 0 */
120 __u8 cmd; /* Command, as defined by the Family */
121 __u8 version; /* Irrelevant, set to 1 */
122 __u16 reserved; /* Reserved, set to 0 */
124 /* TLV attributes follow... */
126 In Classic Netlink :c:member:`nlmsghdr.nlmsg_type` used to identify
127 which operation within the subsystem the message was referring to
128 (e.g. get information about a netdev). Generic Netlink needs to mux
129 multiple subsystems in a single protocol so it uses this field to
130 identify the subsystem, and :c:member:`genlmsghdr.cmd` identifies
131 the operation instead. (See :ref:`res_fam` for
132 information on how to find the Family ID of the subsystem of interest.)
133 Note that the first 16 values (0 - 15) of this field are reserved for
134 control messages both in Classic Netlink and Generic Netlink.
135 See :ref:`nl_msg_type` for more details.
137 There are 3 usual types of message exchanges on a Netlink socket:
139 - performing a single action (``do``);
140 - dumping information (``dump``);
141 - getting asynchronous notifications (``multicast``).
143 Classic Netlink is very flexible and presumably allows other types
144 of exchanges to happen, but in practice those are the three that get
147 Asynchronous notifications are sent by the kernel and received by
148 the user sockets which subscribed to them. ``do`` and ``dump`` requests
149 are initiated by the user. :c:member:`nlmsghdr.nlmsg_flags` should
152 - for ``do``: ``NLM_F_REQUEST | NLM_F_ACK``
153 - for ``dump``: ``NLM_F_REQUEST | NLM_F_ACK | NLM_F_DUMP``
155 :c:member:`nlmsghdr.nlmsg_seq` should be a set to a monotonically
156 increasing value. The value gets echoed back in responses and doesn't
157 matter in practice, but setting it to an increasing value for each
158 message sent is considered good hygiene. The purpose of the field is
159 matching responses to requests. Asynchronous notifications will have
160 :c:member:`nlmsghdr.nlmsg_seq` of ``0``.
162 :c:member:`nlmsghdr.nlmsg_pid` is the Netlink equivalent of an address.
163 This field can be set to ``0`` when talking to the kernel.
164 See :ref:`nlmsg_pid` for the (uncommon) uses of the field.
166 The expected use for :c:member:`genlmsghdr.version` was to allow
167 versioning of the APIs provided by the subsystems. No subsystem to
168 date made significant use of this field, so setting it to ``1`` seems
173 Netlink message types
174 ---------------------
176 As previously mentioned :c:member:`nlmsghdr.nlmsg_type` carries
177 protocol specific values but the first 16 identifiers are reserved
178 (first subsystem specific message type should be equal to
179 ``NLMSG_MIN_TYPE`` which is ``0x10``).
181 There are only 4 Netlink control messages defined:
183 - ``NLMSG_NOOP`` - ignore the message, not used in practice;
184 - ``NLMSG_ERROR`` - carries the return code of an operation;
185 - ``NLMSG_DONE`` - marks the end of a dump;
186 - ``NLMSG_OVERRUN`` - socket buffer has overflown, not used to date.
188 ``NLMSG_ERROR`` and ``NLMSG_DONE`` are of practical importance.
189 They carry return codes for operations. Note that unless
190 the ``NLM_F_ACK`` flag is set on the request Netlink will not respond
191 with ``NLMSG_ERROR`` if there is no error. To avoid having to special-case
192 this quirk it is recommended to always set ``NLM_F_ACK``.
194 The format of ``NLMSG_ERROR`` is described by struct nlmsgerr::
196 ----------------------------------------------
197 | struct nlmsghdr - response header |
198 ----------------------------------------------
200 ----------------------------------------------
201 | struct nlmsghdr - original request header |
202 ----------------------------------------------
203 | ** optionally (1) payload of the request |
204 ----------------------------------------------
205 | ** optionally (2) extended ACK |
206 ----------------------------------------------
208 There are two instances of struct nlmsghdr here, first of the response
209 and second of the request. ``NLMSG_ERROR`` carries the information about
210 the request which led to the error. This could be useful when trying
211 to match requests to responses or re-parse the request to dump it into
214 The payload of the request is not echoed in messages reporting success
215 (``error == 0``) or if ``NETLINK_CAP_ACK`` setsockopt() was set.
217 and perhaps recommended as having to read a copy of every request back
218 from the kernel is rather wasteful. The absence of request payload
219 is indicated by ``NLM_F_CAPPED`` in :c:member:`nlmsghdr.nlmsg_flags`.
221 The second optional element of ``NLMSG_ERROR`` are the extended ACK
222 attributes. See :ref:`ext_ack` for more details. The presence
223 of extended ACK is indicated by ``NLM_F_ACK_TLVS`` in
224 :c:member:`nlmsghdr.nlmsg_flags`.
226 ``NLMSG_DONE`` is simpler, the request is never echoed but the extended
227 ACK attributes may be present::
229 ----------------------------------------------
230 | struct nlmsghdr - response header |
231 ----------------------------------------------
233 ----------------------------------------------
234 | ** optionally extended ACK |
235 ----------------------------------------------
237 Note that some implementations may issue custom ``NLMSG_DONE`` messages
238 in reply to ``do`` action requests. In that case the payload is
239 implementation-specific and may also be absent.
243 Resolving the Family ID
244 -----------------------
246 This section explains how to find the Family ID of a subsystem.
247 It also serves as an example of Generic Netlink communication.
249 Generic Netlink is itself a subsystem exposed via the Generic Netlink API.
250 To avoid a circular dependency Generic Netlink has a statically allocated
251 Family ID (``GENL_ID_CTRL`` which is equal to ``NLMSG_MIN_TYPE``).
252 The Generic Netlink family implements a command used to find out information
253 about other families (``CTRL_CMD_GETFAMILY``).
255 To get information about the Generic Netlink family named for example
256 ``"test1"`` we need to send a message on the previously opened Generic Netlink
257 socket. The message should target the Generic Netlink Family (1), be a
258 ``do`` (2) call to ``CTRL_CMD_GETFAMILY`` (3). A ``dump`` version of this
259 call would make the kernel respond with information about *all* the families
260 it knows about. Last but not least the name of the family in question has
261 to be specified (4) as an attribute with the appropriate type::
265 __u16 nlmsg_type: GENL_ID_CTRL // (1)
266 __u16 nlmsg_flags: NLM_F_REQUEST | NLM_F_ACK // (2)
271 __u8 cmd: CTRL_CMD_GETFAMILY // (3)
272 __u8 version: 2 /* or 1, doesn't matter */
275 struct nlattr: // (4)
277 __u16 nla_type: CTRL_ATTR_FAMILY_NAME
283 The length fields in Netlink (:c:member:`nlmsghdr.nlmsg_len`
284 and :c:member:`nlattr.nla_len`) always *include* the header.
285 Attribute headers in netlink must be aligned to 4 bytes from the start
286 of the message, hence the extra ``\0\0`` after ``CTRL_ATTR_FAMILY_NAME``.
287 The attribute lengths *exclude* the padding.
289 If the family is found kernel will reply with two messages, the response
290 with all the information about the family::
292 /* Message #1 - reply */
295 __u16 nlmsg_type: GENL_ID_CTRL
297 __u32 nlmsg_seq: 1 /* echoed from our request */
298 __u32 nlmsg_pid: 5831 /* The PID of our user space process */
301 __u8 cmd: CTRL_CMD_GETFAMILY
307 __u16 nla_type: CTRL_ATTR_FAMILY_NAME
315 __u16 nla_type: CTRL_ATTR_FAMILY_ID
316 __u16: 123 /* The Family ID we are after */
323 __u16 nla_type: CTRL_ATTR_FAMILY_VERSION
326 /* ... etc, more attributes will follow. */
328 And the error code (success) since ``NLM_F_ACK`` had been set on the request::
330 /* Message #2 - the ACK */
333 __u16 nlmsg_type: NLMSG_ERROR
334 __u16 nlmsg_flags: NLM_F_CAPPED /* There won't be a payload */
335 __u32 nlmsg_seq: 1 /* echoed from our request */
336 __u32 nlmsg_pid: 5831 /* The PID of our user space process */
340 struct nlmsghdr: /* Copy of the request header as we sent it */
342 __u16 nlmsg_type: GENL_ID_CTRL
343 __u16 nlmsg_flags: NLM_F_REQUEST | NLM_F_ACK
347 The order of attributes (struct nlattr) is not guaranteed so the user
348 has to walk the attributes and parse them.
350 Note that Generic Netlink sockets are not associated or bound to a single
351 family. A socket can be used to exchange messages with many different
352 families, selecting the recipient family on message-by-message basis using
353 the :c:member:`nlmsghdr.nlmsg_type` field.
360 Extended ACK controls reporting of additional error/warning TLVs
361 in ``NLMSG_ERROR`` and ``NLMSG_DONE`` messages. To maintain backward
362 compatibility this feature has to be explicitly enabled by setting
363 the ``NETLINK_EXT_ACK`` setsockopt() to ``1``.
365 Types of extended ack attributes are defined in enum nlmsgerr_attrs.
366 The most commonly used attributes are ``NLMSGERR_ATTR_MSG``,
367 ``NLMSGERR_ATTR_OFFS`` and ``NLMSGERR_ATTR_MISS_*``.
369 ``NLMSGERR_ATTR_MSG`` carries a message in English describing
370 the encountered problem. These messages are far more detailed
371 than what can be expressed thru standard UNIX error codes.
373 ``NLMSGERR_ATTR_OFFS`` points to the attribute which caused the problem.
375 ``NLMSGERR_ATTR_MISS_TYPE`` and ``NLMSGERR_ATTR_MISS_NEST``
376 inform about a missing attribute.
378 Extended ACKs can be reported on errors as well as in case of success.
379 The latter should be treated as a warning.
381 Extended ACKs greatly improve the usability of Netlink and should
382 always be enabled, appropriately parsed and reported to the user.
390 Some of the data structures kernel uses for storing objects make
391 it hard to provide an atomic snapshot of all the objects in a dump
392 (without impacting the fast-paths updating them).
394 Kernel may set the ``NLM_F_DUMP_INTR`` flag on any message in a dump
395 (including the ``NLMSG_DONE`` message) if the dump was interrupted and
396 may be inconsistent (e.g. missing objects). User space should retry
397 the dump if it sees the flag set.
402 The basic introspection abilities are enabled by access to the Family
403 object as reported in :ref:`res_fam`. User can query information about
404 the Generic Netlink family, including which operations are supported
405 by the kernel and what attributes the kernel understands.
406 Family information includes the highest ID of an attribute kernel can parse,
407 a separate command (``CTRL_CMD_GETPOLICY``) provides detailed information
408 about supported attributes, including ranges of values the kernel accepts.
410 Querying family information is useful in cases when user space needs
411 to make sure that the kernel has support for a feature before issuing
419 :c:member:`nlmsghdr.nlmsg_pid` is the Netlink equivalent of an address.
420 It is referred to as Port ID, sometimes Process ID because for historical
421 reasons if the application does not select (bind() to) an explicit Port ID
422 kernel will automatically assign it the ID equal to its Process ID
423 (as reported by the getpid() system call).
425 Similarly to the bind() semantics of the TCP/IP network protocols the value
426 of zero means "assign automatically", hence it is common for applications
427 to leave the :c:member:`nlmsghdr.nlmsg_pid` field initialized to ``0``.
429 The field is still used today in rare cases when kernel needs to send
430 a unicast notification. User space application can use bind() to associate
431 its socket with a specific PID, it then communicates its PID to the kernel.
432 This way the kernel can reach the specific user space process.
434 This sort of communication is utilized in UMH (User Mode Helper)-like
435 scenarios when kernel needs to trigger user space processing or ask user
436 space for a policy decision.
438 Multicast notifications
439 -----------------------
441 One of the strengths of Netlink is the ability to send event notifications
442 to user space. This is a unidirectional form of communication (kernel ->
443 user) and does not involve any control messages like ``NLMSG_ERROR`` or
446 For example the Generic Netlink family itself defines a set of multicast
447 notifications about registered families. When a new family is added the
448 sockets subscribed to the notifications will get the following message::
452 __u16 nlmsg_type: GENL_ID_CTRL
458 __u8 cmd: CTRL_CMD_NEWFAMILY
464 __u16 nla_type: CTRL_ATTR_FAMILY_NAME
472 __u16 nla_type: CTRL_ATTR_FAMILY_ID
473 __u16: 123 /* The Family ID we are after */
480 __u16 nla_type: CTRL_ATTR_FAMILY_VERSION
483 /* ... etc, more attributes will follow. */
485 The notification contains the same information as the response
486 to the ``CTRL_CMD_GETFAMILY`` request.
488 The Netlink headers of the notification are mostly 0 and irrelevant.
489 The :c:member:`nlmsghdr.nlmsg_seq` may be either zero or a monotonically
490 increasing notification sequence number maintained by the family.
492 To receive notifications the user socket must subscribe to the relevant
493 notification group. Much like the Family ID, the Group ID for a given
494 multicast group is dynamic and can be found inside the Family information.
495 The ``CTRL_ATTR_MCAST_GROUPS`` attribute contains nests with names
496 (``CTRL_ATTR_MCAST_GRP_NAME``) and IDs (``CTRL_ATTR_MCAST_GRP_ID``) of
499 Once the Group ID is known a setsockopt() call adds the socket to the group:
503 unsigned int group_id;
505 /* .. find the group ID... */
507 setsockopt(fd, SOL_NETLINK, NETLINK_ADD_MEMBERSHIP,
508 &group_id, sizeof(group_id));
510 The socket will now receive notifications.
512 It is recommended to use separate sockets for receiving notifications
513 and sending requests to the kernel. The asynchronous nature of notifications
514 means that they may get mixed in with the responses making the message
515 handling much harder.
520 Netlink sockets are datagram sockets rather than stream sockets,
521 meaning that each message must be received in its entirety by a single
522 recv()/recvmsg() system call. If the buffer provided by the user is too
523 short, the message will be truncated and the ``MSG_TRUNC`` flag set
524 in struct msghdr (struct msghdr is the second argument
525 of the recvmsg() system call, *not* a Netlink header).
527 Upon truncation the remaining part of the message is discarded.
529 Netlink expects that the user buffer will be at least 8kB or a page
530 size of the CPU architecture, whichever is bigger. Particular Netlink
531 families may, however, require a larger buffer. 32kB buffer is recommended
532 for most efficient handling of dumps (larger buffer fits more dumped
533 objects and therefore fewer recvmsg() calls are needed).
540 The main differences between Classic and Generic Netlink are the dynamic
541 allocation of subsystem identifiers and availability of introspection.
542 In theory the protocol does not differ significantly, however, in practice
543 Classic Netlink experimented with concepts which were abandoned in Generic
544 Netlink (really, they usually only found use in a small corner of a single
545 subsystem). This section is meant as an explainer of a few of such concepts,
546 with the explicit goal of giving the Generic Netlink
547 users the confidence to ignore them when reading the uAPI headers.
549 Most of the concepts and examples here refer to the ``NETLINK_ROUTE`` family,
550 which covers much of the configuration of the Linux networking stack.
551 Real documentation of that family, deserves a chapter (or a book) of its own.
556 Netlink refers to subsystems as families. This is a remnant of using
557 sockets and the concept of protocol families, which are part of message
558 demultiplexing in ``NETLINK_ROUTE``.
560 Sadly every layer of encapsulation likes to refer to whatever it's carrying
561 as "families" making the term very confusing:
563 1. AF_NETLINK is a bona fide socket protocol family
564 2. AF_NETLINK's documentation refers to what comes after its own
565 header (struct nlmsghdr) in a message as a "Family Header"
566 3. Generic Netlink is a family for AF_NETLINK (struct genlmsghdr follows
567 struct nlmsghdr), yet it also calls its users "Families".
569 Note that the Generic Netlink Family IDs are in a different "ID space"
570 and overlap with Classic Netlink protocol numbers (e.g. ``NETLINK_CRYPTO``
571 has the Classic Netlink protocol ID of 21 which Generic Netlink will
572 happily allocate to one of its families as well).
577 The ``NETLINK_GET_STRICT_CHK`` socket option enables strict input checking
578 in ``NETLINK_ROUTE``. It was needed because historically kernel did not
579 validate the fields of structures it didn't process. This made it impossible
580 to start using those fields later without risking regressions in applications
581 which initialized them incorrectly or not at all.
583 ``NETLINK_GET_STRICT_CHK`` declares that the application is initializing
584 all fields correctly. It also opts into validating that message does not
585 contain trailing data and requests that kernel rejects attributes with
586 type higher than largest attribute type known to the kernel.
588 ``NETLINK_GET_STRICT_CHK`` is not used outside of ``NETLINK_ROUTE``.
593 Historically Netlink ignored all unknown attributes. The thinking was that
594 it would free the application from having to probe what kernel supports.
595 The application could make a request to change the state and check which
596 parts of the request "stuck".
598 This is no longer the case for new Generic Netlink families and those opting
599 in to strict checking. See enum netlink_validation for validation types
602 Fixed metadata and structures
603 -----------------------------
605 Classic Netlink made liberal use of fixed-format structures within
606 the messages. Messages would commonly have a structure with
607 a considerable number of fields after struct nlmsghdr. It was also
608 common to put structures with multiple members inside attributes,
609 without breaking each member into an attribute of its own.
611 This has caused problems with validation and extensibility and
612 therefore using binary structures is actively discouraged for new
618 ``NETLINK_ROUTE`` categorized requests into 4 types ``NEW``, ``DEL``, ``GET``,
619 and ``SET``. Each object can handle all or some of those requests
620 (objects being netdevs, routes, addresses, qdiscs etc.) Request type
621 is defined by the 2 lowest bits of the message type, so commands for
622 new objects would always be allocated with a stride of 4.
624 Each object would also have its own fixed metadata shared by all request
625 types (e.g. struct ifinfomsg for netdev requests, struct ifaddrmsg for address
626 requests, struct tcmsg for qdisc requests).
628 Even though other protocols and Generic Netlink commands often use
629 the same verbs in their message names (``GET``, ``SET``) the concept
630 of request types did not find wider adoption.
635 ``NLM_F_ECHO`` requests for notifications resulting from the request
636 to be queued onto the requesting socket. This is useful to discover
637 the impact of the request.
639 Note that this feature is not universally implemented.
641 Other request-type-specific flags
642 ---------------------------------
644 Classic Netlink defined various flags for its ``GET``, ``NEW``
645 and ``DEL`` requests in the upper byte of nlmsg_flags in struct nlmsghdr.
646 Since request types have not been generalized the request type specific
647 flags are rarely used (and considered deprecated for new families).
649 For ``GET`` - ``NLM_F_ROOT`` and ``NLM_F_MATCH`` are combined into
650 ``NLM_F_DUMP``, and not used separately. ``NLM_F_ATOMIC`` is never used.
652 For ``DEL`` - ``NLM_F_NONREC`` is only used by nftables and ``NLM_F_BULK``
653 only by FDB some operations.
655 The flags for ``NEW`` are used most commonly in classic Netlink. Unfortunately,
656 the meaning is not crystal clear. The following description is based on the
657 best guess of the intention of the authors, and in practice all families
658 stray from it in one way or another. ``NLM_F_REPLACE`` asks to replace
659 an existing object, if no matching object exists the operation should fail.
660 ``NLM_F_EXCL`` has the opposite semantics and only succeeds if object already
662 ``NLM_F_CREATE`` asks for the object to be created if it does not
663 exist, it can be combined with ``NLM_F_REPLACE`` and ``NLM_F_EXCL``.
665 A comment in the main Netlink uAPI header states::
667 4.4BSD ADD NLM_F_CREATE|NLM_F_EXCL
668 4.4BSD CHANGE NLM_F_REPLACE
670 True CHANGE NLM_F_CREATE|NLM_F_REPLACE
674 which seems to indicate that those flags predate request types.
675 ``NLM_F_REPLACE`` without ``NLM_F_CREATE`` was initially used instead
677 ``NLM_F_EXCL`` without ``NLM_F_CREATE`` was used to check if object exists
678 without creating it, presumably predating ``GET`` commands.
680 ``NLM_F_APPEND`` indicates that if one key can have multiple objects associated
681 with it (e.g. multiple next-hop objects for a route) the new object should be
682 added to the list rather than replacing the entire list.
687 .. kernel-doc:: include/uapi/linux/netlink.h