1 .. SPDX-License-Identifier: GPL-2.0
7 :Author: Masami Hiramatsu <mhiramat@kernel.org>
12 Boot-time tracing allows users to trace boot-time process including
13 device initialization with full features of ftrace including per-event
14 filter and actions, histograms, kprobe-events and synthetic-events,
16 Since kernel command line is not enough to control these complex features,
17 this uses bootconfig file to describe tracing feature programming.
19 Options in the Boot Config
20 ==========================
22 Here is the list of available options list for boot time tracing in
23 boot config file [1]_. All options are under "ftrace." or "kernel."
24 prefix. See kernel parameters for the options which starts
25 with "kernel." prefix [2]_.
27 .. [1] See :ref:`Documentation/admin-guide/bootconfig.rst <bootconfig>`
28 .. [2] See :ref:`Documentation/admin-guide/kernel-parameters.rst <kernelparameters>`
33 Ftrace global options have "kernel." prefix in boot config, which means
34 these options are passed as a part of kernel legacy command line.
37 Output trace-event data on printk buffer too.
39 kernel.dump_on_oops [= MODE]
40 Dump ftrace on Oops. If MODE = 1 or omitted, dump trace buffer
41 on all CPUs. If MODE = 2, dump a buffer on a CPU which kicks Oops.
43 kernel.traceoff_on_warning
44 Stop tracing if WARN_ON() occurs.
46 kernel.fgraph_max_depth = MAX_DEPTH
47 Set MAX_DEPTH to maximum depth of fgraph tracer.
49 kernel.fgraph_filters = FILTER[, FILTER2...]
50 Add fgraph tracing function filters.
52 kernel.fgraph_notraces = FILTER[, FILTER2...]
53 Add fgraph non-tracing function filters.
56 Ftrace Per-instance Options
57 ---------------------------
59 These options can be used for each instance including global ftrace node.
61 ftrace.[instance.INSTANCE.]options = OPT1[, OPT2[...]]
62 Enable given ftrace options.
64 ftrace.[instance.INSTANCE.]tracing_on = 0|1
65 Enable/Disable tracing on this instance when starting boot-time tracing.
66 (you can enable it by the "traceon" event trigger action)
68 ftrace.[instance.INSTANCE.]trace_clock = CLOCK
69 Set given CLOCK to ftrace's trace_clock.
71 ftrace.[instance.INSTANCE.]buffer_size = SIZE
72 Configure ftrace buffer size to SIZE. You can use "KB" or "MB"
75 ftrace.[instance.INSTANCE.]alloc_snapshot
76 Allocate snapshot buffer.
78 ftrace.[instance.INSTANCE.]cpumask = CPUMASK
79 Set CPUMASK as trace cpu-mask.
81 ftrace.[instance.INSTANCE.]events = EVENT[, EVENT2[...]]
82 Enable given events on boot. You can use a wild card in EVENT.
84 ftrace.[instance.INSTANCE.]tracer = TRACER
85 Set TRACER to current tracer on boot. (e.g. function)
87 ftrace.[instance.INSTANCE.]ftrace.filters
88 This will take an array of tracing function filter rules.
90 ftrace.[instance.INSTANCE.]ftrace.notraces
91 This will take an array of NON-tracing function filter rules.
94 Ftrace Per-Event Options
95 ------------------------
97 These options are setting per-event options.
99 ftrace.[instance.INSTANCE.]event.GROUP.EVENT.enable
100 Enable GROUP:EVENT tracing.
102 ftrace.[instance.INSTANCE.]event.GROUP.enable
103 Enable all event tracing within GROUP.
105 ftrace.[instance.INSTANCE.]event.enable
106 Enable all event tracing.
108 ftrace.[instance.INSTANCE.]event.GROUP.EVENT.filter = FILTER
109 Set FILTER rule to the GROUP:EVENT.
111 ftrace.[instance.INSTANCE.]event.GROUP.EVENT.actions = ACTION[, ACTION2[...]]
112 Set ACTIONs to the GROUP:EVENT.
114 ftrace.[instance.INSTANCE.]event.kprobes.EVENT.probes = PROBE[, PROBE2[...]]
115 Defines new kprobe event based on PROBEs. It is able to define
116 multiple probes on one event, but those must have same type of
117 arguments. This option is available only for the event which
118 group name is "kprobes".
120 ftrace.[instance.INSTANCE.]event.synthetic.EVENT.fields = FIELD[, FIELD2[...]]
121 Defines new synthetic event with FIELDs. Each field should be
124 Note that kprobe and synthetic event definitions can be written under
125 instance node, but those are also visible from other instances. So please
126 take care for event name conflict.
128 Ftrace Histogram Options
129 ------------------------
131 Since it is too long to write a histogram action as a string for per-event
132 action option, there are tree-style options under per-event 'hist' subkey
133 for the histogram actions. For the detail of the each parameter,
134 please read the event histogram document [3]_.
136 .. [3] See :ref:`Documentation/trace/histogram.rst <histogram>`
138 ftrace.[instance.INSTANCE.]event.GROUP.EVENT.hist.[N.]keys = KEY1[, KEY2[...]]
139 Set histogram key parameters. (Mandatory)
140 The 'N' is a digit string for the multiple histogram. You can omit it
141 if there is one histogram on the event.
143 ftrace.[instance.INSTANCE.]event.GROUP.EVENT.hist.[N.]values = VAL1[, VAL2[...]]
144 Set histogram value parameters.
146 ftrace.[instance.INSTANCE.]event.GROUP.EVENT.hist.[N.]sort = SORT1[, SORT2[...]]
147 Set histogram sort parameter options.
149 ftrace.[instance.INSTANCE.]event.GROUP.EVENT.hist.[N.]size = NR_ENTRIES
150 Set histogram size (number of entries).
152 ftrace.[instance.INSTANCE.]event.GROUP.EVENT.hist.[N.]name = NAME
155 ftrace.[instance.INSTANCE.]event.GROUP.EVENT.hist.[N.]var.VARIABLE = EXPR
156 Define a new VARIABLE by EXPR expression.
158 ftrace.[instance.INSTANCE.]event.GROUP.EVENT.hist.[N.]<pause|continue|clear>
159 Set histogram control parameter. You can set one of them.
161 ftrace.[instance.INSTANCE.]event.GROUP.EVENT.hist.[N.]onmatch.[M.]event = GROUP.EVENT
162 Set histogram 'onmatch' handler matching event parameter.
163 The 'M' is a digit string for the multiple 'onmatch' handler. You can omit it
164 if there is one 'onmatch' handler on this histogram.
166 ftrace.[instance.INSTANCE.]event.GROUP.EVENT.hist.[N.]onmatch.[M.]trace = EVENT[, ARG1[...]]
167 Set histogram 'trace' action for 'onmatch'.
168 EVENT must be a synthetic event name, and ARG1... are parameters
169 for that event. Mandatory if 'onmatch.event' option is set.
171 ftrace.[instance.INSTANCE.]event.GROUP.EVENT.hist.[N.]onmax.[M.]var = VAR
172 Set histogram 'onmax' handler variable parameter.
174 ftrace.[instance.INSTANCE.]event.GROUP.EVENT.hist.[N.]onchange.[M.]var = VAR
175 Set histogram 'onchange' handler variable parameter.
177 ftrace.[instance.INSTANCE.]event.GROUP.EVENT.hist.[N.]<onmax|onchange>.[M.]save = ARG1[, ARG2[...]]
178 Set histogram 'save' action parameters for 'onmax' or 'onchange' handler.
179 This option or below 'snapshot' option is mandatory if 'onmax.var' or
180 'onchange.var' option is set.
182 ftrace.[instance.INSTANCE.]event.GROUP.EVENT.hist.[N.]<onmax|onchange>.[M.]snapshot
183 Set histogram 'snapshot' action for 'onmax' or 'onchange' handler.
184 This option or above 'save' option is mandatory if 'onmax.var' or
185 'onchange.var' option is set.
187 ftrace.[instance.INSTANCE.]event.GROUP.EVENT.hist.filter = FILTER_EXPR
188 Set histogram filter expression. You don't need 'if' in the FILTER_EXPR.
190 Note that this 'hist' option can conflict with the per-event 'actions'
191 option if the 'actions' option has a histogram action.
197 All boot-time tracing options starting with ``ftrace`` will be enabled at the
198 end of core_initcall. This means you can trace the events from postcore_initcall.
199 Most of the subsystems and architecture dependent drivers will be initialized
200 after that (arch_initcall or subsys_initcall). Thus, you can trace those with
202 If you want to trace events before core_initcall, you can use the options
203 starting with ``kernel``. Some of them will be enabled eariler than the initcall
204 processing (for example,. ``kernel.ftrace=function`` and ``kernel.trace_event``
205 will start before the initcall.)
211 For example, to add filter and actions for each event, define kprobe
212 events, and synthetic events with histogram, write a boot config like
221 probes = "vfs_read $arg1 $arg2"
222 filter = "common_pid < 200"
225 synthetic.initcall_latency {
226 fields = "unsigned long func", "u64 lat"
233 initcall.initcall_start.hist {
235 var.ts0 = common_timestamp.usecs
237 initcall.initcall_finish.hist {
239 var.lat = common_timestamp.usecs - $ts0
241 event = initcall.initcall_start
242 trace = initcall_latency, func, $lat
247 Also, boot-time tracing supports "instance" node, which allows us to run
248 several tracers for different purpose at once. For example, one tracer
249 is for tracing functions starting with "user\_", and others tracing
250 "kernel\_" functions, you can write boot config as below::
255 ftrace.filters = "user_*"
259 ftrace.filters = "kernel_*"
263 The instance node also accepts event nodes so that each instance
264 can customize its event tracing.
266 With the trigger action and kprobes, you can trace function-graph while
267 a function is called. For example, this will trace all function calls in
268 the pci_proc_init()::
272 tracer = function_graph
275 probes = "pci_proc_init"
279 probes = "pci_proc_init%return"
286 This boot-time tracing also supports ftrace kernel parameters via boot
288 For example, following kernel parameters::
290 trace_options=sym-addr trace_event=initcall:* tp_printk trace_buf_size=1M ftrace=function ftrace_filter="vfs*"
292 This can be written in boot config like below::
295 trace_options = sym-addr
296 trace_event = "initcall:*"
300 ftrace_filter = "vfs*"
303 Note that parameters start with "kernel" prefix instead of "ftrace".