1 .. SPDX-License-Identifier: GPL-2.0
3 =======================
4 DAMON-based Reclamation
5 =======================
7 DAMON-based Reclamation (DAMON_RECLAIM) is a static kernel module that aimed to
8 be used for proactive and lightweight reclamation under light memory pressure.
9 It doesn't aim to replace the LRU-list based page_granularity reclamation, but
10 to be selectively used for different level of memory pressure and requirements.
12 Where Proactive Reclamation is Required?
13 ========================================
15 On general memory over-committed systems, proactively reclaiming cold pages
16 helps saving memory and reducing latency spikes that incurred by the direct
17 reclaim of the process or CPU consumption of kswapd, while incurring only
18 minimal performance degradation [1]_ [2]_ .
20 Free Pages Reporting [3]_ based memory over-commit virtualization systems are
21 good example of the cases. In such systems, the guest VMs reports their free
22 memory to host, and the host reallocates the reported memory to other guests.
23 As a result, the memory of the systems are fully utilized. However, the
24 guests could be not so memory-frugal, mainly because some kernel subsystems and
25 user-space applications are designed to use as much memory as available. Then,
26 guests could report only small amount of memory as free to host, results in
27 memory utilization drop of the systems. Running the proactive reclamation in
28 guests could mitigate this problem.
33 DAMON_RECLAIM finds memory regions that didn't accessed for specific time
34 duration and page out. To avoid it consuming too much CPU for the paging out
35 operation, a speed limit can be configured. Under the speed limit, it pages
36 out memory regions that didn't accessed longer time first. System
37 administrators can also configure under what situation this scheme should
38 automatically activated and deactivated with three memory pressure watermarks.
40 Interface: Module Parameters
41 ============================
43 To use this feature, you should first ensure your system is running on a kernel
44 that is built with ``CONFIG_DAMON_RECLAIM=y``.
46 To let sysadmins enable or disable it and tune for the given system,
47 DAMON_RECLAIM utilizes module parameters. That is, you can put
48 ``damon_reclaim.<parameter>=<value>`` on the kernel boot command line or write
49 proper values to ``/sys/module/damon_reclaim/parameters/<parameter>`` files.
51 Below are the description of each parameter.
56 Enable or disable DAMON_RECLAIM.
58 You can enable DAMON_RCLAIM by setting the value of this parameter as ``Y``.
59 Setting it as ``N`` disables DAMON_RECLAIM. Note that DAMON_RECLAIM could do
60 no real monitoring and reclamation due to the watermarks-based activation
61 condition. Refer to below descriptions for the watermarks parameter for this.
66 Make DAMON_RECLAIM reads the input parameters again, except ``enabled``.
68 Input parameters that updated while DAMON_RECLAIM is running are not applied
69 by default. Once this parameter is set as ``Y``, DAMON_RECLAIM reads values
70 of parametrs except ``enabled`` again. Once the re-reading is done, this
71 parameter is set as ``N``. If invalid parameters are found while the
72 re-reading, DAMON_RECLAIM will be disabled.
77 Time threshold for cold memory regions identification in microseconds.
79 If a memory region is not accessed for this or longer time, DAMON_RECLAIM
80 identifies the region as cold, and reclaims it.
82 120 seconds by default.
87 Limit of time for the reclamation in milliseconds.
89 DAMON_RECLAIM tries to use only up to this time within a time window
90 (quota_reset_interval_ms) for trying reclamation of cold pages. This can be
91 used for limiting CPU consumption of DAMON_RECLAIM. If the value is zero, the
99 Limit of size of memory for the reclamation in bytes.
101 DAMON_RECLAIM charges amount of memory which it tried to reclaim within a time
102 window (quota_reset_interval_ms) and makes no more than this limit is tried.
103 This can be used for limiting consumption of CPU and IO. If this value is
104 zero, the limit is disabled.
108 quota_reset_interval_ms
109 -----------------------
111 The time/size quota charge reset interval in milliseconds.
113 The charget reset interval for the quota of time (quota_ms) and size
114 (quota_sz). That is, DAMON_RECLAIM does not try reclamation for more than
115 quota_ms milliseconds or quota_sz bytes within quota_reset_interval_ms
123 Minimal time to wait before checking the watermarks, when DAMON_RECLAIM is
124 enabled but inactive due to its watermarks rule.
129 Free memory rate (per thousand) for the high watermark.
131 If free memory of the system in bytes per thousand bytes is higher than this,
132 DAMON_RECLAIM becomes inactive, so it does nothing but only periodically checks
138 Free memory rate (per thousand) for the middle watermark.
140 If free memory of the system in bytes per thousand bytes is between this and
141 the low watermark, DAMON_RECLAIM becomes active, so starts the monitoring and
147 Free memory rate (per thousand) for the low watermark.
149 If free memory of the system in bytes per thousand bytes is lower than this,
150 DAMON_RECLAIM becomes inactive, so it does nothing but periodically checks the
151 watermarks. In the case, the system falls back to the LRU-list based page
152 granularity reclamation logic.
157 Sampling interval for the monitoring in microseconds.
159 The sampling interval of DAMON for the cold memory monitoring. Please refer to
160 the DAMON documentation (:doc:`usage`) for more detail.
165 Aggregation interval for the monitoring in microseconds.
167 The aggregation interval of DAMON for the cold memory monitoring. Please
168 refer to the DAMON documentation (:doc:`usage`) for more detail.
173 Minimum number of monitoring regions.
175 The minimal number of monitoring regions of DAMON for the cold memory
176 monitoring. This can be used to set lower-bound of the monitoring quality.
177 But, setting this too high could result in increased monitoring overhead.
178 Please refer to the DAMON documentation (:doc:`usage`) for more detail.
183 Maximum number of monitoring regions.
185 The maximum number of monitoring regions of DAMON for the cold memory
186 monitoring. This can be used to set upper-bound of the monitoring overhead.
187 However, setting this too low could result in bad monitoring quality. Please
188 refer to the DAMON documentation (:doc:`usage`) for more detail.
193 Start of target memory region in physical address.
195 The start physical address of memory region that DAMON_RECLAIM will do work
196 against. That is, DAMON_RECLAIM will find cold memory regions in this region
197 and reclaims. By default, biggest System RAM is used as the region.
202 End of target memory region in physical address.
204 The end physical address of memory region that DAMON_RECLAIM will do work
205 against. That is, DAMON_RECLAIM will find cold memory regions in this region
206 and reclaims. By default, biggest System RAM is used as the region.
211 Skip anonymous pages reclamation.
213 If this parameter is set as ``Y``, DAMON_RECLAIM does not reclaim anonymous
214 pages. By default, ``N``.
220 PID of the DAMON thread.
222 If DAMON_RECLAIM is enabled, this becomes the PID of the worker thread. Else,
225 nr_reclaim_tried_regions
226 ------------------------
228 Number of memory regions that tried to be reclaimed by DAMON_RECLAIM.
230 bytes_reclaim_tried_regions
231 ---------------------------
233 Total bytes of memory regions that tried to be reclaimed by DAMON_RECLAIM.
238 Number of memory regions that successfully be reclaimed by DAMON_RECLAIM.
240 bytes_reclaimed_regions
241 -----------------------
243 Total bytes of memory regions that successfully be reclaimed by DAMON_RECLAIM.
248 Number of times that the time/space quota limits have exceeded.
253 Below runtime example commands make DAMON_RECLAIM to find memory regions that
254 not accessed for 30 seconds or more and pages out. The reclamation is limited
255 to be done only up to 1 GiB per second to avoid DAMON_RECLAIM consuming too
256 much CPU time for the paging out operation. It also asks DAMON_RECLAIM to do
257 nothing if the system's free memory rate is more than 50%, but start the real
258 works if it becomes lower than 40%. If DAMON_RECLAIM doesn't make progress and
259 therefore the free memory rate becomes lower than 20%, it asks DAMON_RECLAIM to
260 do nothing again, so that we can fall back to the LRU-list based page
261 granularity reclamation. ::
263 # cd /sys/module/damon_reclaim/parameters
264 # echo 30000000 > min_age
265 # echo $((1 * 1024 * 1024 * 1024)) > quota_sz
266 # echo 1000 > quota_reset_interval_ms
267 # echo 500 > wmarks_high
268 # echo 400 > wmarks_mid
269 # echo 200 > wmarks_low
272 .. [1] https://research.google/pubs/pub48551/
273 .. [2] https://lwn.net/Articles/787611/
274 .. [3] https://www.kernel.org/doc/html/latest/mm/free_page_reporting.html