1 .. SPDX-License-Identifier: GPL-2.0
7 DAMON provides below interfaces for different users.
9 - *DAMON user space tool.*
10 `This <https://github.com/awslabs/damo>`_ is for privileged people such as
11 system administrators who want a just-working human-friendly interface.
12 Using this, users can use the DAMON’s major features in a human-friendly way.
13 It may not be highly tuned for special cases, though. For more detail,
14 please refer to its `usage document
15 <https://github.com/awslabs/damo/blob/next/USAGE.md>`_.
17 :ref:`This <sysfs_interface>` is for privileged user space programmers who
18 want more optimized use of DAMON. Using this, users can use DAMON’s major
19 features by reading from and writing to special sysfs files. Therefore,
20 you can write and use your personalized DAMON sysfs wrapper programs that
21 reads/writes the sysfs files instead of you. The `DAMON user space tool
22 <https://github.com/awslabs/damo>`_ is one example of such programs.
23 - *Kernel Space Programming Interface.*
24 :doc:`This </mm/damon/api>` is for kernel space programmers. Using this,
25 users can utilize every feature of DAMON most flexibly and efficiently by
26 writing kernel space DAMON application programs for you. You can even extend
27 DAMON for various address spaces. For detail, please refer to the interface
28 :doc:`document </mm/damon/api>`.
29 - *debugfs interface. (DEPRECATED!)*
30 :ref:`This <debugfs_interface>` is almost identical to :ref:`sysfs interface
31 <sysfs_interface>`. This is deprecated, so users should move to the
32 :ref:`sysfs interface <sysfs_interface>`. If you depend on this and cannot
33 move, please report your usecase to damon@lists.linux.dev and
41 DAMON sysfs interface is built when ``CONFIG_DAMON_SYSFS`` is defined. It
42 creates multiple directories and files under its sysfs directory,
43 ``<sysfs>/kernel/mm/damon/``. You can control DAMON by writing to and reading
44 from the files under the directory.
46 For a short example, users can monitor the virtual address space of a given
49 # cd /sys/kernel/mm/damon/admin/
50 # echo 1 > kdamonds/nr_kdamonds && echo 1 > kdamonds/0/contexts/nr_contexts
51 # echo vaddr > kdamonds/0/contexts/0/operations
52 # echo 1 > kdamonds/0/contexts/0/targets/nr_targets
53 # echo $(pidof <workload>) > kdamonds/0/contexts/0/targets/0/pid_target
54 # echo on > kdamonds/0/state
59 The files hierarchy of DAMON sysfs interface is shown below. In the below
60 figure, parents-children relations are represented with indentations, each
61 directory is having ``/`` suffix, and files in each directory are separated by
64 /sys/kernel/mm/damon/admin
65 │ kdamonds/nr_kdamonds
67 │ │ │ contexts/nr_contexts
68 │ │ │ │ 0/avail_operations,operations
69 │ │ │ │ │ monitoring_attrs/
70 │ │ │ │ │ │ intervals/sample_us,aggr_us,update_us
71 │ │ │ │ │ │ nr_regions/min,max
72 │ │ │ │ │ targets/nr_targets
73 │ │ │ │ │ │ 0/pid_target
74 │ │ │ │ │ │ │ regions/nr_regions
75 │ │ │ │ │ │ │ │ 0/start,end
78 │ │ │ │ │ schemes/nr_schemes
79 │ │ │ │ │ │ 0/action,apply_interval_us
80 │ │ │ │ │ │ │ access_pattern/
81 │ │ │ │ │ │ │ │ sz/min,max
82 │ │ │ │ │ │ │ │ nr_accesses/min,max
83 │ │ │ │ │ │ │ │ age/min,max
84 │ │ │ │ │ │ │ quotas/ms,bytes,reset_interval_ms
85 │ │ │ │ │ │ │ │ weights/sz_permil,nr_accesses_permil,age_permil
86 │ │ │ │ │ │ │ watermarks/metric,interval_us,high,mid,low
87 │ │ │ │ │ │ │ filters/nr_filters
88 │ │ │ │ │ │ │ │ 0/type,matching,memcg_id
89 │ │ │ │ │ │ │ stats/nr_tried,sz_tried,nr_applied,sz_applied,qt_exceeds
90 │ │ │ │ │ │ │ tried_regions/total_bytes
91 │ │ │ │ │ │ │ │ 0/start,end,nr_accesses,age
100 The root of the DAMON sysfs interface is ``<sysfs>/kernel/mm/damon/``, and it
101 has one directory named ``admin``. The directory contains the files for
102 privileged user space programs' control of DAMON. User space tools or daemons
103 having the root permission could use this directory.
108 Under the ``admin`` directory, one directory, ``kdamonds``, which has files for
109 controlling the kdamonds (refer to
110 :ref:`design <damon_design_execution_model_and_data_structures>` for more
111 details) exists. In the beginning, this directory has only one file,
112 ``nr_kdamonds``. Writing a number (``N``) to the file creates the number of
113 child directories named ``0`` to ``N-1``. Each directory represents each
119 In each kdamond directory, two files (``state`` and ``pid``) and one directory
120 (``contexts``) exist.
122 Reading ``state`` returns ``on`` if the kdamond is currently running, or
123 ``off`` if it is not running. Writing ``on`` or ``off`` makes the kdamond be
124 in the state. Writing ``commit`` to the ``state`` file makes kdamond reads the
125 user inputs in the sysfs files except ``state`` file again. Writing
126 ``update_schemes_stats`` to ``state`` file updates the contents of stats files
127 for each DAMON-based operation scheme of the kdamond. For details of the
128 stats, please refer to :ref:`stats section <sysfs_schemes_stats>`.
130 Writing ``update_schemes_tried_regions`` to ``state`` file updates the
131 DAMON-based operation scheme action tried regions directory for each
132 DAMON-based operation scheme of the kdamond. Writing
133 ``update_schemes_tried_bytes`` to ``state`` file updates only
134 ``.../tried_regions/total_bytes`` files. Writing
135 ``clear_schemes_tried_regions`` to ``state`` file clears the DAMON-based
136 operating scheme action tried regions directory for each DAMON-based operation
137 scheme of the kdamond. For details of the DAMON-based operation scheme action
138 tried regions directory, please refer to :ref:`tried_regions section
139 <sysfs_schemes_tried_regions>`.
141 If the state is ``on``, reading ``pid`` shows the pid of the kdamond thread.
143 ``contexts`` directory contains files for controlling the monitoring contexts
144 that this kdamond will execute.
146 kdamonds/<N>/contexts/
147 ----------------------
149 In the beginning, this directory has only one file, ``nr_contexts``. Writing a
150 number (``N``) to the file creates the number of child directories named as
151 ``0`` to ``N-1``. Each directory represents each monitoring context (refer to
152 :ref:`design <damon_design_execution_model_and_data_structures>` for more
153 details). At the moment, only one context per kdamond is supported, so only
154 ``0`` or ``1`` can be written to the file.
161 In each context directory, two files (``avail_operations`` and ``operations``)
162 and three directories (``monitoring_attrs``, ``targets``, and ``schemes``)
165 DAMON supports multiple types of monitoring operations, including those for
166 virtual address space and the physical address space. You can get the list of
167 available monitoring operations set on the currently running kernel by reading
168 ``avail_operations`` file. Based on the kernel configuration, the file will
169 list some or all of below keywords.
171 - vaddr: Monitor virtual address spaces of specific processes
172 - fvaddr: Monitor fixed virtual address ranges
173 - paddr: Monitor the physical address space of the system
175 Please refer to :ref:`regions sysfs directory <sysfs_regions>` for detailed
176 differences between the operations sets in terms of the monitoring target
179 You can set and get what type of monitoring operations DAMON will use for the
180 context by writing one of the keywords listed in ``avail_operations`` file and
181 reading from the ``operations`` file.
183 .. _sysfs_monitoring_attrs:
185 contexts/<N>/monitoring_attrs/
186 ------------------------------
188 Files for specifying attributes of the monitoring including required quality
189 and efficiency of the monitoring are in ``monitoring_attrs`` directory.
190 Specifically, two directories, ``intervals`` and ``nr_regions`` exist in this
193 Under ``intervals`` directory, three files for DAMON's sampling interval
194 (``sample_us``), aggregation interval (``aggr_us``), and update interval
195 (``update_us``) exist. You can set and get the values in micro-seconds by
196 writing to and reading from the files.
198 Under ``nr_regions`` directory, two files for the lower-bound and upper-bound
199 of DAMON's monitoring regions (``min`` and ``max``, respectively), which
200 controls the monitoring overhead, exist. You can set and get the values by
201 writing to and rading from the files.
203 For more details about the intervals and monitoring regions range, please refer
204 to the Design document (:doc:`/mm/damon/design`).
206 contexts/<N>/targets/
207 ---------------------
209 In the beginning, this directory has only one file, ``nr_targets``. Writing a
210 number (``N``) to the file creates the number of child directories named ``0``
211 to ``N-1``. Each directory represents each monitoring target.
216 In each target directory, one file (``pid_target``) and one directory
219 If you wrote ``vaddr`` to the ``contexts/<N>/operations``, each target should
220 be a process. You can specify the process to DAMON by writing the pid of the
221 process to the ``pid_target`` file.
228 When ``vaddr`` monitoring operations set is being used (``vaddr`` is written to
229 the ``contexts/<N>/operations`` file), DAMON automatically sets and updates the
230 monitoring target regions so that entire memory mappings of target processes
231 can be covered. However, users could want to set the initial monitoring region
232 to specific address ranges.
234 In contrast, DAMON do not automatically sets and updates the monitoring target
235 regions when ``fvaddr`` or ``paddr`` monitoring operations sets are being used
236 (``fvaddr`` or ``paddr`` have written to the ``contexts/<N>/operations``).
237 Therefore, users should set the monitoring target regions by themselves in the
240 For such cases, users can explicitly set the initial monitoring target regions
241 as they want, by writing proper values to the files under this directory.
243 In the beginning, this directory has only one file, ``nr_regions``. Writing a
244 number (``N``) to the file creates the number of child directories named ``0``
245 to ``N-1``. Each directory represents each initial monitoring target region.
250 In each region directory, you will find two files (``start`` and ``end``). You
251 can set and get the start and end addresses of the initial monitoring target
252 region by writing to and reading from the files, respectively.
254 Each region should not overlap with others. ``end`` of directory ``N`` should
255 be equal or smaller than ``start`` of directory ``N+1``.
257 contexts/<N>/schemes/
258 ---------------------
260 The directory for DAMON-based Operation Schemes (:ref:`DAMOS
261 <damon_design_damos>`). Users can get and set the schemes by reading from and
262 writing to files under this directory.
264 In the beginning, this directory has only one file, ``nr_schemes``. Writing a
265 number (``N``) to the file creates the number of child directories named ``0``
266 to ``N-1``. Each directory represents each DAMON-based operation scheme.
271 In each scheme directory, five directories (``access_pattern``, ``quotas``,
272 ``watermarks``, ``filters``, ``stats``, and ``tried_regions``) and two files
273 (``action`` and ``apply_interval``) exist.
275 The ``action`` file is for setting and getting the scheme's :ref:`action
276 <damon_design_damos_action>`. The keywords that can be written to and read
277 from the file and their meaning are as below.
279 Note that support of each action depends on the running DAMON operations set
280 :ref:`implementation <sysfs_contexts>`.
282 - ``willneed``: Call ``madvise()`` for the region with ``MADV_WILLNEED``.
283 Supported by ``vaddr`` and ``fvaddr`` operations set.
284 - ``cold``: Call ``madvise()`` for the region with ``MADV_COLD``.
285 Supported by ``vaddr`` and ``fvaddr`` operations set.
286 - ``pageout``: Call ``madvise()`` for the region with ``MADV_PAGEOUT``.
287 Supported by ``vaddr``, ``fvaddr`` and ``paddr`` operations set.
288 - ``hugepage``: Call ``madvise()`` for the region with ``MADV_HUGEPAGE``.
289 Supported by ``vaddr`` and ``fvaddr`` operations set.
290 - ``nohugepage``: Call ``madvise()`` for the region with ``MADV_NOHUGEPAGE``.
291 Supported by ``vaddr`` and ``fvaddr`` operations set.
292 - ``lru_prio``: Prioritize the region on its LRU lists.
293 Supported by ``paddr`` operations set.
294 - ``lru_deprio``: Deprioritize the region on its LRU lists.
295 Supported by ``paddr`` operations set.
296 - ``stat``: Do nothing but count the statistics.
297 Supported by all operations sets.
299 The ``apply_interval_us`` file is for setting and getting the scheme's
300 :ref:`apply_interval <damon_design_damos>` in microseconds.
302 schemes/<N>/access_pattern/
303 ---------------------------
305 The directory for the target access :ref:`pattern
306 <damon_design_damos_access_pattern>` of the given DAMON-based operation scheme.
308 Under the ``access_pattern`` directory, three directories (``sz``,
309 ``nr_accesses``, and ``age``) each having two files (``min`` and ``max``)
310 exist. You can set and get the access pattern for the given scheme by writing
311 to and reading from the ``min`` and ``max`` files under ``sz``,
312 ``nr_accesses``, and ``age`` directories, respectively. Note that the ``min``
313 and the ``max`` form a closed interval.
318 The directory for the :ref:`quotas <damon_design_damos_quotas>` of the given
319 DAMON-based operation scheme.
321 Under ``quotas`` directory, three files (``ms``, ``bytes``,
322 ``reset_interval_ms``) and one directory (``weights``) having three files
323 (``sz_permil``, ``nr_accesses_permil``, and ``age_permil``) in it exist.
325 You can set the ``time quota`` in milliseconds, ``size quota`` in bytes, and
326 ``reset interval`` in milliseconds by writing the values to the three files,
327 respectively. Then, DAMON tries to use only up to ``time quota`` milliseconds
328 for applying the ``action`` to memory regions of the ``access_pattern``, and to
329 apply the action to only up to ``bytes`` bytes of memory regions within the
330 ``reset_interval_ms``. Setting both ``ms`` and ``bytes`` zero disables the
333 You can also set the :ref:`prioritization weights
334 <damon_design_damos_quotas_prioritization>` for size, access frequency, and age
335 in per-thousand unit by writing the values to the three files under the
336 ``weights`` directory.
338 schemes/<N>/watermarks/
339 -----------------------
341 The directory for the :ref:`watermarks <damon_design_damos_watermarks>` of the
342 given DAMON-based operation scheme.
344 Under the watermarks directory, five files (``metric``, ``interval_us``,
345 ``high``, ``mid``, and ``low``) for setting the metric, the time interval
346 between check of the metric, and the three watermarks exist. You can set and
347 get the five values by writing to the files, respectively.
349 Keywords and meanings of those that can be written to the ``metric`` file are
352 - none: Ignore the watermarks
353 - free_mem_rate: System's free memory rate (per thousand)
355 The ``interval`` should written in microseconds unit.
360 The directory for the :ref:`filters <damon_design_damos_filters>` of the given
361 DAMON-based operation scheme.
363 In the beginning, this directory has only one file, ``nr_filters``. Writing a
364 number (``N``) to the file creates the number of child directories named ``0``
365 to ``N-1``. Each directory represents each filter. The filters are evaluated
366 in the numeric order.
368 Each filter directory contains six files, namely ``type``, ``matcing``,
369 ``memcg_path``, ``addr_start``, ``addr_end``, and ``target_idx``. To ``type``
370 file, you can write one of four special keywords: ``anon`` for anonymous pages,
371 ``memcg`` for specific memory cgroup, ``addr`` for specific address range (an
372 open-ended interval), or ``target`` for specific DAMON monitoring target
373 filtering. In case of the memory cgroup filtering, you can specify the memory
374 cgroup of the interest by writing the path of the memory cgroup from the
375 cgroups mount point to ``memcg_path`` file. In case of the address range
376 filtering, you can specify the start and end address of the range to
377 ``addr_start`` and ``addr_end`` files, respectively. For the DAMON monitoring
378 target filtering, you can specify the index of the target between the list of
379 the DAMON context's monitoring targets list to ``target_idx`` file. You can
380 write ``Y`` or ``N`` to ``matching`` file to filter out pages that does or does
381 not match to the type, respectively. Then, the scheme's action will not be
382 applied to the pages that specified to be filtered out.
384 For example, below restricts a DAMOS action to be applied to only non-anonymous
385 pages of all memory cgroups except ``/having_care_already``.::
387 # echo 2 > nr_filters
388 # # filter out anonymous pages
391 # # further filter out all cgroups except one at '/having_care_already'
393 echo /having_care_already > 1/memcg_path
396 Note that ``anon`` and ``memcg`` filters are currently supported only when
397 ``paddr`` :ref:`implementation <sysfs_contexts>` is being used.
399 Also, memory regions that are filtered out by ``addr`` or ``target`` filters
400 are not counted as the scheme has tried to those, while regions that filtered
401 out by other type filters are counted as the scheme has tried to. The
402 difference is applied to :ref:`stats <damos_stats>` and
403 :ref:`tried regions <sysfs_schemes_tried_regions>`.
405 .. _sysfs_schemes_stats:
410 DAMON counts the total number and bytes of regions that each scheme is tried to
411 be applied, the two numbers for the regions that each scheme is successfully
412 applied, and the total number of the quota limit exceeds. This statistics can
413 be used for online analysis or tuning of the schemes.
415 The statistics can be retrieved by reading the files under ``stats`` directory
416 (``nr_tried``, ``sz_tried``, ``nr_applied``, ``sz_applied``, and
417 ``qt_exceeds``), respectively. The files are not updated in real time, so you
418 should ask DAMON sysfs interface to update the content of the files for the
419 stats by writing a special keyword, ``update_schemes_stats`` to the relevant
420 ``kdamonds/<N>/state`` file.
422 .. _sysfs_schemes_tried_regions:
424 schemes/<N>/tried_regions/
425 --------------------------
427 This directory initially has one file, ``total_bytes``.
429 When a special keyword, ``update_schemes_tried_regions``, is written to the
430 relevant ``kdamonds/<N>/state`` file, DAMON updates the ``total_bytes`` file so
431 that reading it returns the total size of the scheme tried regions, and creates
432 directories named integer starting from ``0`` under this directory. Each
433 directory contains files exposing detailed information about each of the memory
434 region that the corresponding scheme's ``action`` has tried to be applied under
435 this directory, during next :ref:`apply interval <damon_design_damos>` of the
436 corresponding scheme. The information includes address range, ``nr_accesses``,
437 and ``age`` of the region.
439 Writing ``update_schemes_tried_bytes`` to the relevant ``kdamonds/<N>/state``
440 file will only update the ``total_bytes`` file, and will not create the
443 The directories will be removed when another special keyword,
444 ``clear_schemes_tried_regions``, is written to the relevant
445 ``kdamonds/<N>/state`` file.
447 The expected usage of this directory is investigations of schemes' behaviors,
448 and query-like efficient data access monitoring results retrievals. For the
449 latter use case, in particular, users can set the ``action`` as ``stat`` and
450 set the ``access pattern`` as their interested pattern that they want to query.
455 In each region directory, you will find four files (``start``, ``end``,
456 ``nr_accesses``, and ``age``). Reading the files will show the start and end
457 addresses, ``nr_accesses``, and ``age`` of the region that corresponding
458 DAMON-based operation scheme ``action`` has tried to be applied.
463 Below commands applies a scheme saying "If a memory region of size in [4KiB,
464 8KiB] is showing accesses per aggregate interval in [0, 5] for aggregate
465 interval in [10, 20], page out the region. For the paging out, use only up to
466 10ms per second, and also don't page out more than 1GiB per second. Under the
467 limitation, page out memory regions having longer age first. Also, check the
468 free memory rate of the system every 5 seconds, start the monitoring and paging
469 out when the free memory rate becomes lower than 50%, but stop it if the free
470 memory rate becomes larger than 60%, or lower than 30%". ::
472 # cd <sysfs>/kernel/mm/damon/admin
473 # # populate directories
474 # echo 1 > kdamonds/nr_kdamonds; echo 1 > kdamonds/0/contexts/nr_contexts;
475 # echo 1 > kdamonds/0/contexts/0/schemes/nr_schemes
476 # cd kdamonds/0/contexts/0/schemes/0
477 # # set the basic access pattern and the action
478 # echo 4096 > access_pattern/sz/min
479 # echo 8192 > access_pattern/sz/max
480 # echo 0 > access_pattern/nr_accesses/min
481 # echo 5 > access_pattern/nr_accesses/max
482 # echo 10 > access_pattern/age/min
483 # echo 20 > access_pattern/age/max
484 # echo pageout > action
486 # echo 10 > quotas/ms
487 # echo $((1024*1024*1024)) > quotas/bytes
488 # echo 1000 > quotas/reset_interval_ms
490 # echo free_mem_rate > watermarks/metric
491 # echo 5000000 > watermarks/interval_us
492 # echo 600 > watermarks/high
493 # echo 500 > watermarks/mid
494 # echo 300 > watermarks/low
496 Please note that it's highly recommended to use user space tools like `damo
497 <https://github.com/awslabs/damo>`_ rather than manually reading and writing
498 the files as above. Above is only for an example.
502 Tracepoints for Monitoring Results
503 ==================================
505 Users can get the monitoring results via the :ref:`tried_regions
506 <sysfs_schemes_tried_regions>`. The interface is useful for getting a
507 snapshot, but it could be inefficient for fully recording all the monitoring
508 results. For the purpose, two trace points, namely ``damon:damon_aggregated``
509 and ``damon:damos_before_apply``, are provided. ``damon:damon_aggregated``
510 provides the whole monitoring results, while ``damon:damos_before_apply``
511 provides the monitoring results for regions that each DAMON-based Operation
512 Scheme (:ref:`DAMOS <damon_design_damos>`) is gonna be applied. Hence,
513 ``damon:damos_before_apply`` is more useful for recording internal behavior of
514 DAMOS, or DAMOS target access
515 :ref:`pattern <damon_design_damos_access_pattern>` based query-like efficient
516 monitoring results recording.
518 While the monitoring is turned on, you could record the tracepoint events and
519 show results using tracepoint supporting tools like ``perf``. For example::
521 # echo on > monitor_on
522 # perf record -e damon:damon_aggregated &
524 # kill 9 $(pidof perf)
525 # echo off > monitor_on
527 kdamond.0 46568 [027] 79357.842179: damon:damon_aggregated: target_id=0 nr_regions=11 122509119488-135708762112: 0 864
530 Each line of the perf script output represents each monitoring region. The
531 first five fields are as usual other tracepoint outputs. The sixth field
532 (``target_id=X``) shows the ide of the monitoring target of the region. The
533 seventh field (``nr_regions=X``) shows the total number of monitoring regions
534 for the target. The eighth field (``X-Y:``) shows the start (``X``) and end
535 (``Y``) addresses of the region in bytes. The ninth field (``X``) shows the
536 ``nr_accesses`` of the region (refer to
537 :ref:`design <damon_design_region_based_sampling>` for more details of the
538 counter). Finally the tenth field (``X``) shows the ``age`` of the region
539 (refer to :ref:`design <damon_design_age_tracking>` for more details of the
542 If the event was ``damon:damos_beofre_apply``, the ``perf script`` output would
543 be somewhat like below::
545 kdamond.0 47293 [000] 80801.060214: damon:damos_before_apply: ctx_idx=0 scheme_idx=0 target_idx=0 nr_regions=11 121932607488-135128711168: 0 136
548 Each line of the output represents each monitoring region that each DAMON-based
549 Operation Scheme was about to be applied at the traced time. The first five
550 fields are as usual. It shows the index of the DAMON context (``ctx_idx=X``)
551 of the scheme in the list of the contexts of the context's kdamond, the index
552 of the scheme (``scheme_idx=X``) in the list of the schemes of the context, in
553 addition to the output of ``damon_aggregated`` tracepoint.
556 .. _debugfs_interface:
558 debugfs Interface (DEPRECATED!)
559 ===============================
565 DAMON debugfs interface is deprecated, so users should move to the
566 :ref:`sysfs interface <sysfs_interface>`. If you depend on this and cannot
567 move, please report your usecase to damon@lists.linux.dev and
570 DAMON exports eight files, ``attrs``, ``target_ids``, ``init_regions``,
571 ``schemes``, ``monitor_on``, ``kdamond_pid``, ``mk_contexts`` and
572 ``rm_contexts`` under its debugfs directory, ``<debugfs>/damon/``.
578 Users can get and set the ``sampling interval``, ``aggregation interval``,
579 ``update interval``, and min/max number of monitoring target regions by
580 reading from and writing to the ``attrs`` file. To know about the monitoring
581 attributes in detail, please refer to the :doc:`/mm/damon/design`. For
582 example, below commands set those values to 5 ms, 100 ms, 1,000 ms, 10 and
583 1000, and then check it again::
586 # echo 5000 100000 1000000 10 1000 > attrs
588 5000 100000 1000000 10 1000
594 Some types of address spaces supports multiple monitoring target. For example,
595 the virtual memory address spaces monitoring can have multiple processes as the
596 monitoring targets. Users can set the targets by writing relevant id values of
597 the targets to, and get the ids of the current targets by reading from the
598 ``target_ids`` file. In case of the virtual address spaces monitoring, the
599 values should be pids of the monitoring target processes. For example, below
600 commands set processes having pids 42 and 4242 as the monitoring targets and
604 # echo 42 4242 > target_ids
608 Users can also monitor the physical memory address space of the system by
609 writing a special keyword, "``paddr\n``" to the file. Because physical address
610 space monitoring doesn't support multiple targets, reading the file will show a
611 fake value, ``42``, as below::
614 # echo paddr > target_ids
618 Note that setting the target ids doesn't start the monitoring.
621 Initial Monitoring Target Regions
622 ---------------------------------
624 In case of the virtual address space monitoring, DAMON automatically sets and
625 updates the monitoring target regions so that entire memory mappings of target
626 processes can be covered. However, users can want to limit the monitoring
627 region to specific address ranges, such as the heap, the stack, or specific
628 file-mapped area. Or, some users can know the initial access pattern of their
629 workloads and therefore want to set optimal initial regions for the 'adaptive
632 In contrast, DAMON do not automatically sets and updates the monitoring target
633 regions in case of physical memory monitoring. Therefore, users should set the
634 monitoring target regions by themselves.
636 In such cases, users can explicitly set the initial monitoring target regions
637 as they want, by writing proper values to the ``init_regions`` file. The input
638 should be a sequence of three integers separated by white spaces that represent
639 one region in below form.::
641 <target idx> <start address> <end address>
643 The ``target idx`` should be the index of the target in ``target_ids`` file,
644 starting from ``0``, and the regions should be passed in address order. For
645 example, below commands will set a couple of address ranges, ``1-100`` and
646 ``100-200`` as the initial monitoring target region of pid 42, which is the
647 first one (index ``0``) in ``target_ids``, and another couple of address
648 ranges, ``20-40`` and ``50-100`` as that of pid 4242, which is the second one
649 (index ``1``) in ``target_ids``.::
657 1 50 100" > init_regions
659 Note that this sets the initial monitoring target regions only. In case of
660 virtual memory monitoring, DAMON will automatically updates the boundary of the
661 regions after one ``update interval``. Therefore, users should set the
662 ``update interval`` large enough in this case, if they don't want the
669 Users can get and set the DAMON-based operation :ref:`schemes
670 <damon_design_damos>` by reading from and writing to ``schemes`` debugfs file.
671 Reading the file also shows the statistics of each scheme. To the file, each
672 of the schemes should be represented in each line in below form::
674 <target access pattern> <action> <quota> <watermarks>
676 You can disable schemes by simply writing an empty string to the file.
678 Target Access Pattern
679 ~~~~~~~~~~~~~~~~~~~~~
681 The target access :ref:`pattern <damon_design_damos_access_pattern>` of the
682 scheme. The ``<target access pattern>`` is constructed with three ranges in
685 min-size max-size min-acc max-acc min-age max-age
687 Specifically, bytes for the size of regions (``min-size`` and ``max-size``),
688 number of monitored accesses per aggregate interval for access frequency
689 (``min-acc`` and ``max-acc``), number of aggregate intervals for the age of
690 regions (``min-age`` and ``max-age``) are specified. Note that the ranges are
696 The ``<action>`` is a predefined integer for memory management :ref:`actions
697 <damon_design_damos_action>`. The supported numbers and their meanings are as
700 - 0: Call ``madvise()`` for the region with ``MADV_WILLNEED``. Ignored if
701 ``target`` is ``paddr``.
702 - 1: Call ``madvise()`` for the region with ``MADV_COLD``. Ignored if
703 ``target`` is ``paddr``.
704 - 2: Call ``madvise()`` for the region with ``MADV_PAGEOUT``.
705 - 3: Call ``madvise()`` for the region with ``MADV_HUGEPAGE``. Ignored if
706 ``target`` is ``paddr``.
707 - 4: Call ``madvise()`` for the region with ``MADV_NOHUGEPAGE``. Ignored if
708 ``target`` is ``paddr``.
709 - 5: Do nothing but count the statistics
714 Users can set the :ref:`quotas <damon_design_damos_quotas>` of the given scheme
715 via the ``<quota>`` in below form::
717 <ms> <sz> <reset interval> <priority weights>
719 This makes DAMON to try to use only up to ``<ms>`` milliseconds for applying
720 the action to memory regions of the ``target access pattern`` within the
721 ``<reset interval>`` milliseconds, and to apply the action to only up to
722 ``<sz>`` bytes of memory regions within the ``<reset interval>``. Setting both
723 ``<ms>`` and ``<sz>`` zero disables the quota limits.
725 For the :ref:`prioritization <damon_design_damos_quotas_prioritization>`, users
726 can set the weights for the three properties in ``<priority weights>`` in below
729 <size weight> <access frequency weight> <age weight>
734 Users can specify :ref:`watermarks <damon_design_damos_watermarks>` of the
735 given scheme via ``<watermarks>`` in below form::
737 <metric> <check interval> <high mark> <middle mark> <low mark>
739 ``<metric>`` is a predefined integer for the metric to be checked. The
740 supported numbers and their meanings are as below.
742 - 0: Ignore the watermarks
743 - 1: System's free memory rate (per thousand)
745 The value of the metric is checked every ``<check interval>`` microseconds.
747 If the value is higher than ``<high mark>`` or lower than ``<low mark>``, the
748 scheme is deactivated. If the value is lower than ``<mid mark>``, the scheme
756 It also counts the total number and bytes of regions that each scheme is tried
757 to be applied, the two numbers for the regions that each scheme is successfully
758 applied, and the total number of the quota limit exceeds. This statistics can
759 be used for online analysis or tuning of the schemes.
761 The statistics can be shown by reading the ``schemes`` file. Reading the file
762 will show each scheme you entered in each line, and the five numbers for the
763 statistics will be added at the end of each line.
768 Below commands applies a scheme saying "If a memory region of size in [4KiB,
769 8KiB] is showing accesses per aggregate interval in [0, 5] for aggregate
770 interval in [10, 20], page out the region. For the paging out, use only up to
771 10ms per second, and also don't page out more than 1GiB per second. Under the
772 limitation, page out memory regions having longer age first. Also, check the
773 free memory rate of the system every 5 seconds, start the monitoring and paging
774 out when the free memory rate becomes lower than 50%, but stop it if the free
775 memory rate becomes larger than 60%, or lower than 30%".::
778 # scheme="4096 8192 0 5 10 20 2" # target access pattern and action
779 # scheme+=" 10 $((1024*1024*1024)) 1000" # quotas
780 # scheme+=" 0 0 100" # prioritization weights
781 # scheme+=" 1 5000000 600 500 300" # watermarks
782 # echo "$scheme" > schemes
788 Setting the files as described above doesn't incur effect unless you explicitly
789 start the monitoring. You can start, stop, and check the current status of the
790 monitoring by writing to and reading from the ``monitor_on`` file. Writing
791 ``on`` to the file starts the monitoring of the targets with the attributes.
792 Writing ``off`` to the file stops those. DAMON also stops if every target
793 process is terminated. Below example commands turn on, off, and check the
797 # echo on > monitor_on
798 # echo off > monitor_on
802 Please note that you cannot write to the above-mentioned debugfs files while
803 the monitoring is turned on. If you write to the files while DAMON is running,
804 an error code such as ``-EBUSY`` will be returned.
807 Monitoring Thread PID
808 ---------------------
810 DAMON does requested monitoring with a kernel thread called ``kdamond``. You
811 can get the pid of the thread by reading the ``kdamond_pid`` file. When the
812 monitoring is turned off, reading the file returns ``none``. ::
819 # echo on > monitor_on
824 Using Multiple Monitoring Threads
825 ---------------------------------
827 One ``kdamond`` thread is created for each monitoring context. You can create
828 and remove monitoring contexts for multiple ``kdamond`` required use case using
829 the ``mk_contexts`` and ``rm_contexts`` files.
831 Writing the name of the new context to the ``mk_contexts`` file creates a
832 directory of the name on the DAMON debugfs directory. The directory will have
833 DAMON debugfs files for the context. ::
837 # ls: cannot access 'foo': No such file or directory
838 # echo foo > mk_contexts
840 # attrs init_regions kdamond_pid schemes target_ids
842 If the context is not needed anymore, you can remove it and the corresponding
843 directory by putting the name of the context to the ``rm_contexts`` file. ::
845 # echo foo > rm_contexts
847 # ls: cannot access 'foo': No such file or directory
849 Note that ``mk_contexts``, ``rm_contexts``, and ``monitor_on`` files are in the
projects / linux-modified.git / blob