1 .. SPDX-License-Identifier: GPL-2.0
2 .. _xfs_online_fsck_design:
5 Mapping of heading styles within this document:
6 Heading 1 uses "====" above and below
14 Sections are manually numbered because apparently that's what everyone
17 ======================
18 XFS Online Fsck Design
19 ======================
21 This document captures the design of the online filesystem check feature for
23 The purpose of this document is threefold:
25 - To help kernel distributors understand exactly what the XFS online fsck
26 feature is, and issues about which they should be aware.
28 - To help people reading the code to familiarize themselves with the relevant
29 concepts and design points before they start digging into the code.
31 - To help developers maintaining the system by capturing the reasons
32 supporting higher level decision making.
34 As the online fsck code is merged, the links in this document to topic branches
35 will be replaced with links to code.
37 This document is licensed under the terms of the GNU Public License, v2.
38 The primary author is Darrick J. Wong.
40 This design document is split into seven parts.
41 Part 1 defines what fsck tools are and the motivations for writing a new one.
42 Parts 2 and 3 present a high level overview of how online fsck process works
43 and how it is tested to ensure correct functionality.
44 Part 4 discusses the user interface and the intended usage modes of the new
46 Parts 5 and 6 show off the high level components and how they fit together, and
47 then present case studies of how each repair function actually works.
48 Part 7 sums up what has been discussed so far and speculates about what else
49 might be built atop online fsck.
51 .. contents:: Table of Contents
54 1. What is a Filesystem Check?
55 ==============================
57 A Unix filesystem has four main responsibilities:
59 - Provide a hierarchy of names through which application programs can associate
60 arbitrary blobs of data for any length of time,
62 - Virtualize physical storage media across those names, and
64 - Retrieve the named data blobs at any time.
66 - Examine resource usage.
68 Metadata directly supporting these functions (e.g. files, directories, space
69 mappings) are sometimes called primary metadata.
70 Secondary metadata (e.g. reverse mapping and directory parent pointers) support
71 operations internal to the filesystem, such as internal consistency checking
73 Summary metadata, as the name implies, condense information contained in
74 primary metadata for performance reasons.
76 The filesystem check (fsck) tool examines all the metadata in a filesystem
78 In addition to looking for obvious metadata corruptions, fsck also
79 cross-references different types of metadata records with each other to look
81 People do not like losing data, so most fsck tools also contains some ability
82 to correct any problems found.
83 As a word of caution -- the primary goal of most Linux fsck tools is to restore
84 the filesystem metadata to a consistent state, not to maximize the data
86 That precedent will not be challenged here.
88 Filesystems of the 20th century generally lacked any redundancy in the ondisk
89 format, which means that fsck can only respond to errors by erasing files until
90 errors are no longer detected.
91 More recent filesystem designs contain enough redundancy in their metadata that
92 it is now possible to regenerate data structures when non-catastrophic errors
93 occur; this capability aids both strategies.
95 +--------------------------------------------------------------------------+
97 +--------------------------------------------------------------------------+
98 | System administrators avoid data loss by increasing the number of |
99 | separate storage systems through the creation of backups; and they avoid |
100 | downtime by increasing the redundancy of each storage system through the |
101 | creation of RAID arrays. |
102 | fsck tools address only the first problem. |
103 +--------------------------------------------------------------------------+
105 TLDR; Show Me the Code!
106 -----------------------
108 Code is posted to the kernel.org git trees as follows:
109 `kernel changes <https://git.kernel.org/pub/scm/linux/kernel/git/djwong/xfs-linux.git/log/?h=repair-symlink>`_,
110 `userspace changes <https://git.kernel.org/pub/scm/linux/kernel/git/djwong/xfsprogs-dev.git/log/?h=scrub-media-scan-service>`_, and
111 `QA test changes <https://git.kernel.org/pub/scm/linux/kernel/git/djwong/xfstests-dev.git/log/?h=repair-dirs>`_.
112 Each kernel patchset adding an online repair function will use the same branch
113 name across the kernel, xfsprogs, and fstests git repos.
118 The online fsck tool described here will be the third tool in the history of
119 XFS (on Linux) to check and repair filesystems.
120 Two programs precede it:
122 The first program, ``xfs_check``, was created as part of the XFS debugger
123 (``xfs_db``) and can only be used with unmounted filesystems.
124 It walks all metadata in the filesystem looking for inconsistencies in the
125 metadata, though it lacks any ability to repair what it finds.
126 Due to its high memory requirements and inability to repair things, this
127 program is now deprecated and will not be discussed further.
129 The second program, ``xfs_repair``, was created to be faster and more robust
130 than the first program.
131 Like its predecessor, it can only be used with unmounted filesystems.
132 It uses extent-based in-memory data structures to reduce memory consumption,
133 and tries to schedule readahead IO appropriately to reduce I/O waiting time
134 while it scans the metadata of the entire filesystem.
135 The most important feature of this tool is its ability to respond to
136 inconsistencies in file metadata and directory tree by erasing things as needed
137 to eliminate problems.
138 Space usage metadata are rebuilt from the observed file metadata.
143 The current XFS tools leave several problems unsolved:
145 1. **User programs** suddenly **lose access** to the filesystem when unexpected
146 shutdowns occur as a result of silent corruptions in the metadata.
147 These occur **unpredictably** and often without warning.
149 2. **Users** experience a **total loss of service** during the recovery period
150 after an **unexpected shutdown** occurs.
152 3. **Users** experience a **total loss of service** if the filesystem is taken
153 offline to **look for problems** proactively.
155 4. **Data owners** cannot **check the integrity** of their stored data without
157 This may expose them to substantial billing costs when a linear media scan
158 performed by the storage system administrator might suffice.
160 5. **System administrators** cannot **schedule** a maintenance window to deal
161 with corruptions if they **lack the means** to assess filesystem health
162 while the filesystem is online.
164 6. **Fleet monitoring tools** cannot **automate periodic checks** of filesystem
165 health when doing so requires **manual intervention** and downtime.
167 7. **Users** can be tricked into **doing things they do not desire** when
168 malicious actors **exploit quirks of Unicode** to place misleading names
171 Given this definition of the problems to be solved and the actors who would
172 benefit, the proposed solution is a third fsck tool that acts on a running
175 This new third program has three components: an in-kernel facility to check
176 metadata, an in-kernel facility to repair metadata, and a userspace driver
177 program to drive fsck activity on a live filesystem.
178 ``xfs_scrub`` is the name of the driver program.
179 The rest of this document presents the goals and use cases of the new fsck
180 tool, describes its major design points in connection to those goals, and
181 discusses the similarities and differences with existing tools.
183 +--------------------------------------------------------------------------+
185 +--------------------------------------------------------------------------+
186 | Throughout this document, the existing offline fsck tool can also be |
187 | referred to by its current name "``xfs_repair``". |
188 | The userspace driver program for the new online fsck tool can be |
189 | referred to as "``xfs_scrub``". |
190 | The kernel portion of online fsck that validates metadata is called |
191 | "online scrub", and portion of the kernel that fixes metadata is called |
193 +--------------------------------------------------------------------------+
195 The naming hierarchy is broken up into objects known as directories and files
196 and the physical space is split into pieces known as allocation groups.
197 Sharding enables better performance on highly parallel systems and helps to
198 contain the damage when corruptions occur.
199 The division of the filesystem into principal objects (allocation groups and
200 inodes) means that there are ample opportunities to perform targeted checks and
201 repairs on a subset of the filesystem.
203 While this is going on, other parts continue processing IO requests.
204 Even if a piece of filesystem metadata can only be regenerated by scanning the
205 entire system, the scan can still be done in the background while other file
208 In summary, online fsck takes advantage of resource sharding and redundant
209 metadata to enable targeted checking and repair operations while the system
211 This capability will be coupled to automatic system management so that
212 autonomous self-healing of XFS maximizes service availability.
214 2. Theory of Operation
215 ======================
217 Because it is necessary for online fsck to lock and scan live metadata objects,
218 online fsck consists of three separate code components.
219 The first is the userspace driver program ``xfs_scrub``, which is responsible
220 for identifying individual metadata items, scheduling work items for them,
221 reacting to the outcomes appropriately, and reporting results to the system
223 The second and third are in the kernel, which implements functions to check
224 and repair each type of online fsck work item.
226 +------------------------------------------------------------------+
228 +------------------------------------------------------------------+
229 | For brevity, this document shortens the phrase "online fsck work |
230 | item" to "scrub item". |
231 +------------------------------------------------------------------+
233 Scrub item types are delineated in a manner consistent with the Unix design
234 philosophy, which is to say that each item should handle one aspect of a
235 metadata structure, and handle it well.
240 In principle, online fsck should be able to check and to repair everything that
241 the offline fsck program can handle.
242 However, online fsck cannot be running 100% of the time, which means that
243 latent errors may creep in after a scrub completes.
244 If these errors cause the next mount to fail, offline fsck is the only
246 This limitation means that maintenance of the offline fsck tool will continue.
247 A second limitation of online fsck is that it must follow the same resource
248 sharing and lock acquisition rules as the regular filesystem.
249 This means that scrub cannot take *any* shortcuts to save time, because doing
250 so could lead to concurrency problems.
251 In other words, online fsck is not a complete replacement for offline fsck, and
252 a complete run of online fsck may take longer than online fsck.
253 However, both of these limitations are acceptable tradeoffs to satisfy the
254 different motivations of online fsck, which are to **minimize system downtime**
255 and to **increase predictability of operation**.
262 The userspace driver program ``xfs_scrub`` splits the work of checking and
263 repairing an entire filesystem into seven phases.
264 Each phase concentrates on checking specific types of scrub items and depends
265 on the success of all previous phases.
266 The seven phases are as follows:
268 1. Collect geometry information about the mounted filesystem and computer,
269 discover the online fsck capabilities of the kernel, and open the
270 underlying storage devices.
272 2. Check allocation group metadata, all realtime volume metadata, and all quota
274 Each metadata structure is scheduled as a separate scrub item.
275 If corruption is found in the inode header or inode btree and ``xfs_scrub``
276 is permitted to perform repairs, then those scrub items are repaired to
278 Repairs are implemented by using the information in the scrub item to
279 resubmit the kernel scrub call with the repair flag enabled; this is
280 discussed in the next section.
281 Optimizations and all other repairs are deferred to phase 4.
283 3. Check all metadata of every file in the filesystem.
284 Each metadata structure is also scheduled as a separate scrub item.
285 If repairs are needed and ``xfs_scrub`` is permitted to perform repairs,
286 and there were no problems detected during phase 2, then those scrub items
287 are repaired immediately.
288 Optimizations, deferred repairs, and unsuccessful repairs are deferred to
291 4. All remaining repairs and scheduled optimizations are performed during this
292 phase, if the caller permits them.
293 Before starting repairs, the summary counters are checked and any necessary
294 repairs are performed so that subsequent repairs will not fail the resource
295 reservation step due to wildly incorrect summary counters.
296 Unsuccessful repairs are requeued as long as forward progress on repairs is
297 made somewhere in the filesystem.
298 Free space in the filesystem is trimmed at the end of phase 4 if the
301 5. By the start of this phase, all primary and secondary filesystem metadata
303 Summary counters such as the free space counts and quota resource counts
304 are checked and corrected.
305 Directory entry names and extended attribute names are checked for
306 suspicious entries such as control characters or confusing Unicode sequences
309 6. If the caller asks for a media scan, read all allocated and written data
310 file extents in the filesystem.
311 The ability to use hardware-assisted data file integrity checking is new
312 to online fsck; neither of the previous tools have this capability.
313 If media errors occur, they will be mapped to the owning files and reported.
315 7. Re-check the summary counters and presents the caller with a summary of
316 space usage and file counts.
318 This allocation of responsibilities will be :ref:`revisited <scrubcheck>`
319 later in this document.
321 Steps for Each Scrub Item
322 -------------------------
324 The kernel scrub code uses a three-step strategy for checking and repairing
325 the one aspect of a metadata object represented by a scrub item:
327 1. The scrub item of interest is checked for corruptions; opportunities for
328 optimization; and for values that are directly controlled by the system
329 administrator but look suspicious.
330 If the item is not corrupt or does not need optimization, resource are
331 released and the positive scan results are returned to userspace.
332 If the item is corrupt or could be optimized but the caller does not permit
333 this, resources are released and the negative scan results are returned to
335 Otherwise, the kernel moves on to the second step.
337 2. The repair function is called to rebuild the data structure.
338 Repair functions generally choose rebuild a structure from other metadata
339 rather than try to salvage the existing structure.
340 If the repair fails, the scan results from the first step are returned to
342 Otherwise, the kernel moves on to the third step.
344 3. In the third step, the kernel runs the same checks over the new metadata
345 item to assess the efficacy of the repairs.
346 The results of the reassessment are returned to userspace.
348 Classification of Metadata
349 --------------------------
351 Each type of metadata object (and therefore each type of scrub item) is
352 classified as follows:
357 Metadata structures in this category should be most familiar to filesystem
358 users either because they are directly created by the user or they index
359 objects created by the user
360 Most filesystem objects fall into this class:
362 - Free space and reference count information
364 - Inode records and indexes
366 - Storage mapping information for file data
370 - Extended attributes
376 Scrub obeys the same rules as regular filesystem accesses for resource and lock
379 Primary metadata objects are the simplest for scrub to process.
380 The principal filesystem object (either an allocation group or an inode) that
381 owns the item being scrubbed is locked to guard against concurrent updates.
382 The check function examines every record associated with the type for obvious
383 errors and cross-references healthy records against other metadata to look for
385 Repairs for this class of scrub item are simple, since the repair function
386 starts by holding all the resources acquired in the previous step.
387 The repair function scans available metadata as needed to record all the
388 observations needed to complete the structure.
389 Next, it stages the observations in a new ondisk structure and commits it
390 atomically to complete the repair.
391 Finally, the storage from the old data structure are carefully reaped.
393 Because ``xfs_scrub`` locks a primary object for the duration of the repair,
394 this is effectively an offline repair operation performed on a subset of the
396 This minimizes the complexity of the repair code because it is not necessary to
397 handle concurrent updates from other threads, nor is it necessary to access
398 any other part of the filesystem.
399 As a result, indexed structures can be rebuilt very quickly, and programs
400 trying to access the damaged structure will be blocked until repairs complete.
401 The only infrastructure needed by the repair code are the staging area for
402 observations and a means to write new structures to disk.
403 Despite these limitations, the advantage that online repair holds is clear:
404 targeted work on individual shards of the filesystem avoids total loss of
407 This mechanism is described in section 2.1 ("Off-Line Algorithm") of
408 V. Srinivasan and M. J. Carey, `"Performance of On-Line Index Construction
409 Algorithms" <https://minds.wisconsin.edu/bitstream/handle/1793/59524/TR1047.pdf>`_,
410 *Extending Database Technology*, pp. 293-309, 1992.
412 Most primary metadata repair functions stage their intermediate results in an
413 in-memory array prior to formatting the new ondisk structure, which is very
414 similar to the list-based algorithm discussed in section 2.3 ("List-Based
415 Algorithms") of Srinivasan.
416 However, any data structure builder that maintains a resource lock for the
417 duration of the repair is *always* an offline algorithm.
419 .. _secondary_metadata:
424 Metadata structures in this category reflect records found in primary metadata,
425 but are only needed for online fsck or for reorganization of the filesystem.
427 Secondary metadata include:
429 - Reverse mapping information
431 - Directory parent pointers
433 This class of metadata is difficult for scrub to process because scrub attaches
434 to the secondary object but needs to check primary metadata, which runs counter
435 to the usual order of resource acquisition.
436 Frequently, this means that full filesystems scans are necessary to rebuild the
438 Check functions can be limited in scope to reduce runtime.
439 Repairs, however, require a full scan of primary metadata, which can take a
440 long time to complete.
441 Under these conditions, ``xfs_scrub`` cannot lock resources for the entire
442 duration of the repair.
444 Instead, repair functions set up an in-memory staging structure to store
446 Depending on the requirements of the specific repair function, the staging
447 index will either have the same format as the ondisk structure or a design
448 specific to that repair function.
449 The next step is to release all locks and start the filesystem scan.
450 When the repair scanner needs to record an observation, the staging data are
451 locked long enough to apply the update.
452 While the filesystem scan is in progress, the repair function hooks the
453 filesystem so that it can apply pending filesystem updates to the staging
455 Once the scan is done, the owning object is re-locked, the live data is used to
456 write a new ondisk structure, and the repairs are committed atomically.
457 The hooks are disabled and the staging staging area is freed.
458 Finally, the storage from the old data structure are carefully reaped.
460 Introducing concurrency helps online repair avoid various locking problems, but
461 comes at a high cost to code complexity.
462 Live filesystem code has to be hooked so that the repair function can observe
464 The staging area has to become a fully functional parallel structure so that
465 updates can be merged from the hooks.
466 Finally, the hook, the filesystem scan, and the inode locking model must be
467 sufficiently well integrated that a hook event can decide if a given update
468 should be applied to the staging structure.
470 In theory, the scrub implementation could apply these same techniques for
471 primary metadata, but doing so would make it massively more complex and less
473 Programs attempting to access the damaged structures are not blocked from
474 operation, which may cause application failure or an unplanned filesystem
477 Inspiration for the secondary metadata repair strategy was drawn from section
478 2.4 of Srinivasan above, and sections 2 ("NSF: Inded Build Without Side-File")
479 and 3.1.1 ("Duplicate Key Insert Problem") in C. Mohan, `"Algorithms for
480 Creating Indexes for Very Large Tables Without Quiescing Updates"
481 <https://dl.acm.org/doi/10.1145/130283.130337>`_, 1992.
483 The sidecar index mentioned above bears some resemblance to the side file
484 method mentioned in Srinivasan and Mohan.
485 Their method consists of an index builder that extracts relevant record data to
486 build the new structure as quickly as possible; and an auxiliary structure that
487 captures all updates that would be committed to the index by other threads were
488 the new index already online.
489 After the index building scan finishes, the updates recorded in the side file
490 are applied to the new index.
491 To avoid conflicts between the index builder and other writer threads, the
492 builder maintains a publicly visible cursor that tracks the progress of the
493 scan through the record space.
494 To avoid duplication of work between the side file and the index builder, side
495 file updates are elided when the record ID for the update is greater than the
496 cursor position within the record ID space.
498 To minimize changes to the rest of the codebase, XFS online repair keeps the
499 replacement index hidden until it's completely ready to go.
500 In other words, there is no attempt to expose the keyspace of the new index
501 while repair is running.
502 The complexity of such an approach would be very high and perhaps more
503 appropriate to building *new* indices.
505 **Future Work Question**: Can the full scan and live update code used to
506 facilitate a repair also be used to implement a comprehensive check?
508 *Answer*: In theory, yes. Check would be much stronger if each scrub function
509 employed these live scans to build a shadow copy of the metadata and then
510 compared the shadow records to the ondisk records.
511 However, doing that is a fair amount more work than what the checking functions
513 The live scans and hooks were developed much later.
514 That in turn increases the runtime of those scrub functions.
519 Metadata structures in this last category summarize the contents of primary
521 These are often used to speed up resource usage queries, and are many times
522 smaller than the primary metadata which they represent.
524 Examples of summary information include:
526 - Summary counts of free space and inodes
528 - File link counts from directories
530 - Quota resource usage counts
532 Check and repair require full filesystem scans, but resource and lock
533 acquisition follow the same paths as regular filesystem accesses.
535 The superblock summary counters have special requirements due to the underlying
536 implementation of the incore counters, and will be treated separately.
537 Check and repair of the other types of summary counters (quota resource counts
538 and file link counts) employ the same filesystem scanning and hooking
539 techniques as outlined above, but because the underlying data are sets of
540 integer counters, the staging data need not be a fully functional mirror of the
543 Inspiration for quota and file link count repair strategies were drawn from
544 sections 2.12 ("Online Index Operations") through 2.14 ("Incremental View
545 Maintenance") of G. Graefe, `"Concurrent Queries and Updates in Summary Views
547 <http://www.odbms.org/wp-content/uploads/2014/06/Increment-locks.pdf>`_, 2011.
549 Since quotas are non-negative integer counts of resource usage, online
550 quotacheck can use the incremental view deltas described in section 2.14 to
551 track pending changes to the block and inode usage counts in each transaction,
552 and commit those changes to a dquot side file when the transaction commits.
553 Delta tracking is necessary for dquots because the index builder scans inodes,
554 whereas the data structure being rebuilt is an index of dquots.
555 Link count checking combines the view deltas and commit step into one because
556 it sets attributes of the objects being scanned instead of writing them to a
557 separate data structure.
558 Each online fsck function will be discussed as case studies later in this
564 During the development of online fsck, several risk factors were identified
565 that may make the feature unsuitable for certain distributors and users.
566 Steps can be taken to mitigate or eliminate those risks, though at a cost to
569 - **Decreased performance**: Adding metadata indices to the filesystem
570 increases the time cost of persisting changes to disk, and the reverse space
571 mapping and directory parent pointers are no exception.
572 System administrators who require the maximum performance can disable the
573 reverse mapping features at format time, though this choice dramatically
574 reduces the ability of online fsck to find inconsistencies and repair them.
576 - **Incorrect repairs**: As with all software, there might be defects in the
577 software that result in incorrect repairs being written to the filesystem.
578 Systematic fuzz testing (detailed in the next section) is employed by the
579 authors to find bugs early, but it might not catch everything.
580 The kernel build system provides Kconfig options (``CONFIG_XFS_ONLINE_SCRUB``
581 and ``CONFIG_XFS_ONLINE_REPAIR``) to enable distributors to choose not to
583 The xfsprogs build system has a configure option (``--enable-scrub=no``) that
584 disables building of the ``xfs_scrub`` binary, though this is not a risk
585 mitigation if the kernel functionality remains enabled.
587 - **Inability to repair**: Sometimes, a filesystem is too badly damaged to be
589 If the keyspaces of several metadata indices overlap in some manner but a
590 coherent narrative cannot be formed from records collected, then the repair
592 To reduce the chance that a repair will fail with a dirty transaction and
593 render the filesystem unusable, the online repair functions have been
594 designed to stage and validate all new records before committing the new
597 - **Misbehavior**: Online fsck requires many privileges -- raw IO to block
598 devices, opening files by handle, ignoring Unix discretionary access control,
599 and the ability to perform administrative changes.
600 Running this automatically in the background scares people, so the systemd
601 background service is configured to run with only the privileges required.
602 Obviously, this cannot address certain problems like the kernel crashing or
603 deadlocking, but it should be sufficient to prevent the scrub process from
604 escaping and reconfiguring the system.
605 The cron job does not have this protection.
607 - **Fuzz Kiddiez**: There are many people now who seem to think that running
608 automated fuzz testing of ondisk artifacts to find mischievous behavior and
609 spraying exploit code onto the public mailing list for instant zero-day
610 disclosure is somehow of some social benefit.
611 In the view of this author, the benefit is realized only when the fuzz
612 operators help to **fix** the flaws, but this opinion apparently is not
613 widely shared among security "researchers".
614 The XFS maintainers' continuing ability to manage these events presents an
615 ongoing risk to the stability of the development process.
616 Automated testing should front-load some of the risk while the feature is
617 considered EXPERIMENTAL.
619 Many of these risks are inherent to software programming.
620 Despite this, it is hoped that this new functionality will prove useful in
621 reducing unexpected downtime.
626 As stated before, fsck tools have three main goals:
628 1. Detect inconsistencies in the metadata;
630 2. Eliminate those inconsistencies; and
632 3. Minimize further loss of data.
634 Demonstrations of correct operation are necessary to build users' confidence
635 that the software behaves within expectations.
636 Unfortunately, it was not really feasible to perform regular exhaustive testing
637 of every aspect of a fsck tool until the introduction of low-cost virtual
638 machines with high-IOPS storage.
639 With ample hardware availability in mind, the testing strategy for the online
640 fsck project involves differential analysis against the existing fsck tools and
641 systematic testing of every attribute of every type of metadata object.
642 Testing can be split into four major categories, as discussed below.
644 Integrated Testing with fstests
645 -------------------------------
647 The primary goal of any free software QA effort is to make testing as
648 inexpensive and widespread as possible to maximize the scaling advantages of
650 In other words, testing should maximize the breadth of filesystem configuration
651 scenarios and hardware setups.
652 This improves code quality by enabling the authors of online fsck to find and
653 fix bugs early, and helps developers of new features to find integration
654 issues earlier in their development effort.
656 The Linux filesystem community shares a common QA testing suite,
657 `fstests <https://git.kernel.org/pub/scm/fs/xfs/xfstests-dev.git/>`_, for
658 functional and regression testing.
659 Even before development work began on online fsck, fstests (when run on XFS)
660 would run both the ``xfs_check`` and ``xfs_repair -n`` commands on the test and
661 scratch filesystems between each test.
662 This provides a level of assurance that the kernel and the fsck tools stay in
663 alignment about what constitutes consistent metadata.
664 During development of the online checking code, fstests was modified to run
665 ``xfs_scrub -n`` between each test to ensure that the new checking code
666 produces the same results as the two existing fsck tools.
668 To start development of online repair, fstests was modified to run
669 ``xfs_repair`` to rebuild the filesystem's metadata indices between tests.
670 This ensures that offline repair does not crash, leave a corrupt filesystem
671 after it exists, or trigger complaints from the online check.
672 This also established a baseline for what can and cannot be repaired offline.
673 To complete the first phase of development of online repair, fstests was
674 modified to be able to run ``xfs_scrub`` in a "force rebuild" mode.
675 This enables a comparison of the effectiveness of online repair as compared to
676 the existing offline repair tools.
678 General Fuzz Testing of Metadata Blocks
679 ---------------------------------------
681 XFS benefits greatly from having a very robust debugging tool, ``xfs_db``.
683 Before development of online fsck even began, a set of fstests were created
684 to test the rather common fault that entire metadata blocks get corrupted.
685 This required the creation of fstests library code that can create a filesystem
686 containing every possible type of metadata object.
687 Next, individual test cases were created to create a test filesystem, identify
688 a single block of a specific type of metadata object, trash it with the
689 existing ``blocktrash`` command in ``xfs_db``, and test the reaction of a
690 particular metadata validation strategy.
692 This earlier test suite enabled XFS developers to test the ability of the
693 in-kernel validation functions and the ability of the offline fsck tool to
694 detect and eliminate the inconsistent metadata.
695 This part of the test suite was extended to cover online fsck in exactly the
698 In other words, for a given fstests filesystem configuration:
700 * For each metadata object existing on the filesystem:
702 * Write garbage to it
704 * Test the reactions of:
706 1. The kernel verifiers to stop obviously bad metadata
707 2. Offline repair (``xfs_repair``) to detect and fix
708 3. Online repair (``xfs_scrub``) to detect and fix
710 Targeted Fuzz Testing of Metadata Records
711 -----------------------------------------
713 The testing plan for online fsck includes extending the existing fs testing
714 infrastructure to provide a much more powerful facility: targeted fuzz testing
715 of every metadata field of every metadata object in the filesystem.
716 ``xfs_db`` can modify every field of every metadata structure in every
717 block in the filesystem to simulate the effects of memory corruption and
719 Given that fstests already contains the ability to create a filesystem
720 containing every metadata format known to the filesystem, ``xfs_db`` can be
721 used to perform exhaustive fuzz testing!
723 For a given fstests filesystem configuration:
725 * For each metadata object existing on the filesystem...
727 * For each record inside that metadata object...
729 * For each field inside that record...
731 * For each conceivable type of transformation that can be applied to a bit field...
735 3. Toggle the most significant bit
736 4. Toggle the middle bit
737 5. Toggle the least significant bit
738 6. Add a small quantity
739 7. Subtract a small quantity
740 8. Randomize the contents
742 * ...test the reactions of:
744 1. The kernel verifiers to stop obviously bad metadata
745 2. Offline checking (``xfs_repair -n``)
746 3. Offline repair (``xfs_repair``)
747 4. Online checking (``xfs_scrub -n``)
748 5. Online repair (``xfs_scrub``)
749 6. Both repair tools (``xfs_scrub`` and then ``xfs_repair`` if online repair doesn't succeed)
751 This is quite the combinatoric explosion!
753 Fortunately, having this much test coverage makes it easy for XFS developers to
754 check the responses of XFS' fsck tools.
755 Since the introduction of the fuzz testing framework, these tests have been
756 used to discover incorrect repair code and missing functionality for entire
757 classes of metadata objects in ``xfs_repair``.
758 The enhanced testing was used to finalize the deprecation of ``xfs_check`` by
759 confirming that ``xfs_repair`` could detect at least as many corruptions as
762 These tests have been very valuable for ``xfs_scrub`` in the same ways -- they
763 allow the online fsck developers to compare online fsck against offline fsck,
764 and they enable XFS developers to find deficiencies in the code base.
766 Proposed patchsets include
767 `general fuzzer improvements
768 <https://git.kernel.org/pub/scm/linux/kernel/git/djwong/xfstests-dev.git/log/?h=fuzzer-improvements>`_,
770 <https://git.kernel.org/pub/scm/linux/kernel/git/djwong/xfstests-dev.git/log/?h=fuzz-baseline>`_,
771 and `improvements in fuzz testing comprehensiveness
772 <https://git.kernel.org/pub/scm/linux/kernel/git/djwong/xfstests-dev.git/log/?h=more-fuzz-testing>`_.
777 A unique requirement to online fsck is the ability to operate on a filesystem
778 concurrently with regular workloads.
779 Although it is of course impossible to run ``xfs_scrub`` with *zero* observable
780 impact on the running system, the online repair code should never introduce
781 inconsistencies into the filesystem metadata, and regular workloads should
782 never notice resource starvation.
783 To verify that these conditions are being met, fstests has been enhanced in
786 * For each scrub item type, create a test to exercise checking that item type
787 while running ``fsstress``.
788 * For each scrub item type, create a test to exercise repairing that item type
789 while running ``fsstress``.
790 * Race ``fsstress`` and ``xfs_scrub -n`` to ensure that checking the whole
791 filesystem doesn't cause problems.
792 * Race ``fsstress`` and ``xfs_scrub`` in force-rebuild mode to ensure that
793 force-repairing the whole filesystem doesn't cause problems.
794 * Race ``xfs_scrub`` in check and force-repair mode against ``fsstress`` while
795 freezing and thawing the filesystem.
796 * Race ``xfs_scrub`` in check and force-repair mode against ``fsstress`` while
797 remounting the filesystem read-only and read-write.
798 * The same, but running ``fsx`` instead of ``fsstress``. (Not done yet?)
800 Success is defined by the ability to run all of these tests without observing
801 any unexpected filesystem shutdowns due to corrupted metadata, kernel hang
802 check warnings, or any other sort of mischief.
804 Proposed patchsets include `general stress testing
805 <https://git.kernel.org/pub/scm/linux/kernel/git/djwong/xfstests-dev.git/log/?h=race-scrub-and-mount-state-changes>`_
806 and the `evolution of existing per-function stress testing
807 <https://git.kernel.org/pub/scm/linux/kernel/git/djwong/xfstests-dev.git/log/?h=refactor-scrub-stress>`_.
812 The primary user of online fsck is the system administrator, just like offline
814 Online fsck presents two modes of operation to administrators:
815 A foreground CLI process for online fsck on demand, and a background service
816 that performs autonomous checking and repair.
821 For administrators who want the absolute freshest information about the
822 metadata in a filesystem, ``xfs_scrub`` can be run as a foreground process on
824 The program checks every piece of metadata in the filesystem while the
825 administrator waits for the results to be reported, just like the existing
827 Both tools share a ``-n`` option to perform a read-only scan, and a ``-v``
828 option to increase the verbosity of the information reported.
830 A new feature of ``xfs_scrub`` is the ``-x`` option, which employs the error
831 correction capabilities of the hardware to check data file contents.
832 The media scan is not enabled by default because it may dramatically increase
833 program runtime and consume a lot of bandwidth on older storage hardware.
835 The output of a foreground invocation is captured in the system log.
837 The ``xfs_scrub_all`` program walks the list of mounted filesystems and
838 initiates ``xfs_scrub`` for each of them in parallel.
839 It serializes scans for any filesystems that resolve to the same top level
840 kernel block device to prevent resource overconsumption.
845 To reduce the workload of system administrators, the ``xfs_scrub`` package
846 provides a suite of `systemd <https://systemd.io/>`_ timers and services that
847 run online fsck automatically on weekends by default.
848 The background service configures scrub to run with as little privilege as
849 possible, the lowest CPU and IO priority, and in a CPU-constrained single
851 This can be tuned by the systemd administrator at any time to suit the latency
852 and throughput requirements of customer workloads.
854 The output of the background service is also captured in the system log.
855 If desired, reports of failures (either due to inconsistencies or mere runtime
856 errors) can be emailed automatically by setting the ``EMAIL_ADDR`` environment
857 variable in the following service files:
859 * ``xfs_scrub_fail@.service``
860 * ``xfs_scrub_media_fail@.service``
861 * ``xfs_scrub_all_fail.service``
863 The decision to enable the background scan is left to the system administrator.
864 This can be done by enabling either of the following services:
866 * ``xfs_scrub_all.timer`` on systemd systems
867 * ``xfs_scrub_all.cron`` on non-systemd systems
869 This automatic weekly scan is configured out of the box to perform an
870 additional media scan of all file data once per month.
871 This is less foolproof than, say, storing file data block checksums, but much
872 more performant if application software provides its own integrity checking,
873 redundancy can be provided elsewhere above the filesystem, or the storage
874 device's integrity guarantees are deemed sufficient.
876 The systemd unit file definitions have been subjected to a security audit
877 (as of systemd 249) to ensure that the xfs_scrub processes have as little
878 access to the rest of the system as possible.
879 This was performed via ``systemd-analyze security``, after which privileges
880 were restricted to the minimum required, sandboxing was set up to the maximal
881 extent possible with sandboxing and system call filtering; and access to the
882 filesystem tree was restricted to the minimum needed to start the program and
883 access the filesystem being scanned.
884 The service definition files restrict CPU usage to 80% of one CPU core, and
885 apply as nice of a priority to IO and CPU scheduling as possible.
886 This measure was taken to minimize delays in the rest of the filesystem.
887 No such hardening has been performed for the cron job.
890 `Enabling the xfs_scrub background service
891 <https://git.kernel.org/pub/scm/linux/kernel/git/djwong/xfsprogs-dev.git/log/?h=scrub-media-scan-service>`_.
896 XFS caches a summary of each filesystem's health status in memory.
897 The information is updated whenever ``xfs_scrub`` is run, or whenever
898 inconsistencies are detected in the filesystem metadata during regular
900 System administrators should use the ``health`` command of ``xfs_spaceman`` to
901 download this information into a human-readable format.
902 If problems have been observed, the administrator can schedule a reduced
903 service window to run the online repair tool to correct the problem.
904 Failing that, the administrator can decide to schedule a maintenance window to
905 run the traditional offline repair tool to correct the problem.
907 **Future Work Question**: Should the health reporting integrate with the new
908 inotify fs error notification system?
909 Would it be helpful for sysadmins to have a daemon to listen for corruption
910 notifications and initiate a repair?
912 *Answer*: These questions remain unanswered, but should be a part of the
913 conversation with early adopters and potential downstream users of XFS.
915 Proposed patchsets include
916 `wiring up health reports to correction returns
917 <https://git.kernel.org/pub/scm/linux/kernel/git/djwong/xfs-linux.git/log/?h=corruption-health-reports>`_
919 `preservation of sickness info during memory reclaim
920 <https://git.kernel.org/pub/scm/linux/kernel/git/djwong/xfs-linux.git/log/?h=indirect-health-reporting>`_.
922 5. Kernel Algorithms and Data Structures
923 ========================================
925 This section discusses the key algorithms and data structures of the kernel
926 code that provide the ability to check and repair metadata while the system
928 The first chapters in this section reveal the pieces that provide the
929 foundation for checking metadata.
930 The remainder of this section presents the mechanisms through which XFS
933 Self Describing Metadata
934 ------------------------
936 Starting with XFS version 5 in 2012, XFS updated the format of nearly every
937 ondisk block header to record a magic number, a checksum, a universally
938 "unique" identifier (UUID), an owner code, the ondisk address of the block,
939 and a log sequence number.
940 When loading a block buffer from disk, the magic number, UUID, owner, and
941 ondisk address confirm that the retrieved block matches the specific owner of
942 the current filesystem, and that the information contained in the block is
943 supposed to be found at the ondisk address.
944 The first three components enable checking tools to disregard alleged metadata
945 that doesn't belong to the filesystem, and the fourth component enables the
946 filesystem to detect lost writes.
948 Whenever a file system operation modifies a block, the change is submitted
949 to the log as part of a transaction.
950 The log then processes these transactions marking them done once they are
951 safely persisted to storage.
952 The logging code maintains the checksum and the log sequence number of the last
953 transactional update.
954 Checksums are useful for detecting torn writes and other discrepancies that can
955 be introduced between the computer and its storage devices.
956 Sequence number tracking enables log recovery to avoid applying out of date
957 log updates to the filesystem.
959 These two features improve overall runtime resiliency by providing a means for
960 the filesystem to detect obvious corruption when reading metadata blocks from
961 disk, but these buffer verifiers cannot provide any consistency checking
962 between metadata structures.
964 For more information, please see the documentation for
965 Documentation/filesystems/xfs-self-describing-metadata.rst
970 The original design of XFS (circa 1993) is an improvement upon 1980s Unix
972 In those days, storage density was expensive, CPU time was scarce, and
973 excessive seek time could kill performance.
974 For performance reasons, filesystem authors were reluctant to add redundancy to
975 the filesystem, even at the cost of data integrity.
976 Filesystems designers in the early 21st century choose different strategies to
977 increase internal redundancy -- either storing nearly identical copies of
978 metadata, or more space-efficient encoding techniques.
980 For XFS, a different redundancy strategy was chosen to modernize the design:
981 a secondary space usage index that maps allocated disk extents back to their
983 By adding a new index, the filesystem retains most of its ability to scale
984 well to heavily threaded workloads involving large datasets, since the primary
985 file metadata (the directory tree, the file block map, and the allocation
986 groups) remain unchanged.
987 Like any system that improves redundancy, the reverse-mapping feature increases
988 overhead costs for space mapping activities.
989 However, it has two critical advantages: first, the reverse index is key to
990 enabling online fsck and other requested functionality such as free space
991 defragmentation, better media failure reporting, and filesystem shrinking.
992 Second, the different ondisk storage format of the reverse mapping btree
993 defeats device-level deduplication because the filesystem requires real
996 +--------------------------------------------------------------------------+
998 +--------------------------------------------------------------------------+
999 | A criticism of adding the secondary index is that it does nothing to |
1000 | improve the robustness of user data storage itself. |
1001 | This is a valid point, but adding a new index for file data block |
1002 | checksums increases write amplification by turning data overwrites into |
1003 | copy-writes, which age the filesystem prematurely. |
1004 | In keeping with thirty years of precedent, users who want file data |
1005 | integrity can supply as powerful a solution as they require. |
1006 | As for metadata, the complexity of adding a new secondary index of space |
1007 | usage is much less than adding volume management and storage device |
1008 | mirroring to XFS itself. |
1009 | Perfection of RAID and volume management are best left to existing |
1010 | layers in the kernel. |
1011 +--------------------------------------------------------------------------+
1013 The information captured in a reverse space mapping record is as follows:
1017 struct xfs_rmap_irec {
1018 xfs_agblock_t rm_startblock; /* extent start block */
1019 xfs_extlen_t rm_blockcount; /* extent length */
1020 uint64_t rm_owner; /* extent owner */
1021 uint64_t rm_offset; /* offset within the owner */
1022 unsigned int rm_flags; /* state flags */
1025 The first two fields capture the location and size of the physical space,
1026 in units of filesystem blocks.
1027 The owner field tells scrub which metadata structure or file inode have been
1028 assigned this space.
1029 For space allocated to files, the offset field tells scrub where the space was
1030 mapped within the file fork.
1031 Finally, the flags field provides extra information about the space usage --
1032 is this an attribute fork extent? A file mapping btree extent? Or an
1033 unwritten data extent?
1035 Online filesystem checking judges the consistency of each primary metadata
1036 record by comparing its information against all other space indices.
1037 The reverse mapping index plays a key role in the consistency checking process
1038 because it contains a centralized alternate copy of all space allocation
1040 Program runtime and ease of resource acquisition are the only real limits to
1041 what online checking can consult.
1042 For example, a file data extent mapping can be checked against:
1044 * The absence of an entry in the free space information.
1045 * The absence of an entry in the inode index.
1046 * The absence of an entry in the reference count data if the file is not
1047 marked as having shared extents.
1048 * The correspondence of an entry in the reverse mapping information.
1050 There are several observations to make about reverse mapping indices:
1052 1. Reverse mappings can provide a positive affirmation of correctness if any of
1053 the above primary metadata are in doubt.
1054 The checking code for most primary metadata follows a path similar to the
1057 2. Proving the consistency of secondary metadata with the primary metadata is
1058 difficult because that requires a full scan of all primary space metadata,
1059 which is very time intensive.
1060 For example, checking a reverse mapping record for a file extent mapping
1061 btree block requires locking the file and searching the entire btree to
1063 Instead, scrub relies on rigorous cross-referencing during the primary space
1064 mapping structure checks.
1066 3. Consistency scans must use non-blocking lock acquisition primitives if the
1067 required locking order is not the same order used by regular filesystem
1069 For example, if the filesystem normally takes a file ILOCK before taking
1070 the AGF buffer lock but scrub wants to take a file ILOCK while holding
1071 an AGF buffer lock, scrub cannot block on that second acquisition.
1072 This means that forward progress during this part of a scan of the reverse
1073 mapping data cannot be guaranteed if system load is heavy.
1075 In summary, reverse mappings play a key role in reconstruction of primary
1077 The details of how these records are staged, written to disk, and committed
1078 into the filesystem are covered in subsequent sections.
1080 Checking and Cross-Referencing
1081 ------------------------------
1083 The first step of checking a metadata structure is to examine every record
1084 contained within the structure and its relationship with the rest of the
1086 XFS contains multiple layers of checking to try to prevent inconsistent
1087 metadata from wreaking havoc on the system.
1088 Each of these layers contributes information that helps the kernel to make
1089 three decisions about the health of a metadata structure:
1091 - Is a part of this structure obviously corrupt (``XFS_SCRUB_OFLAG_CORRUPT``) ?
1092 - Is this structure inconsistent with the rest of the system
1093 (``XFS_SCRUB_OFLAG_XCORRUPT``) ?
1094 - Is there so much damage around the filesystem that cross-referencing is not
1095 possible (``XFS_SCRUB_OFLAG_XFAIL``) ?
1096 - Can the structure be optimized to improve performance or reduce the size of
1097 metadata (``XFS_SCRUB_OFLAG_PREEN``) ?
1098 - Does the structure contain data that is not inconsistent but deserves review
1099 by the system administrator (``XFS_SCRUB_OFLAG_WARNING``) ?
1101 The following sections describe how the metadata scrubbing process works.
1103 Metadata Buffer Verification
1104 ````````````````````````````
1106 The lowest layer of metadata protection in XFS are the metadata verifiers built
1107 into the buffer cache.
1108 These functions perform inexpensive internal consistency checking of the block
1109 itself, and answer these questions:
1111 - Does the block belong to this filesystem?
1113 - Does the block belong to the structure that asked for the read?
1114 This assumes that metadata blocks only have one owner, which is always true
1117 - Is the type of data stored in the block within a reasonable range of what
1120 - Does the physical location of the block match the location it was read from?
1122 - Does the block checksum match the data?
1124 The scope of the protections here are very limited -- verifiers can only
1125 establish that the filesystem code is reasonably free of gross corruption bugs
1126 and that the storage system is reasonably competent at retrieval.
1127 Corruption problems observed at runtime cause the generation of health reports,
1128 failed system calls, and in the extreme case, filesystem shutdowns if the
1129 corrupt metadata force the cancellation of a dirty transaction.
1131 Every online fsck scrubbing function is expected to read every ondisk metadata
1132 block of a structure in the course of checking the structure.
1133 Corruption problems observed during a check are immediately reported to
1134 userspace as corruption; during a cross-reference, they are reported as a
1135 failure to cross-reference once the full examination is complete.
1136 Reads satisfied by a buffer already in cache (and hence already verified)
1137 bypass these checks.
1139 Internal Consistency Checks
1140 ```````````````````````````
1142 After the buffer cache, the next level of metadata protection is the internal
1143 record verification code built into the filesystem.
1144 These checks are split between the buffer verifiers, the in-filesystem users of
1145 the buffer cache, and the scrub code itself, depending on the amount of higher
1146 level context required.
1147 The scope of checking is still internal to the block.
1148 These higher level checking functions answer these questions:
1150 - Does the type of data stored in the block match what scrub is expecting?
1152 - Does the block belong to the owning structure that asked for the read?
1154 - If the block contains records, do the records fit within the block?
1156 - If the block tracks internal free space information, is it consistent with
1159 - Are the records contained inside the block free of obvious corruptions?
1161 Record checks in this category are more rigorous and more time-intensive.
1162 For example, block pointers and inumbers are checked to ensure that they point
1163 within the dynamically allocated parts of an allocation group and within
1165 Names are checked for invalid characters, and flags are checked for invalid
1167 Other record attributes are checked for sensible values.
1168 Btree records spanning an interval of the btree keyspace are checked for
1169 correct order and lack of mergeability (except for file fork mappings).
1170 For performance reasons, regular code may skip some of these checks unless
1171 debugging is enabled or a write is about to occur.
1172 Scrub functions, of course, must check all possible problems.
1174 Validation of Userspace-Controlled Record Attributes
1175 ````````````````````````````````````````````````````
1177 Various pieces of filesystem metadata are directly controlled by userspace.
1178 Because of this nature, validation work cannot be more precise than checking
1179 that a value is within the possible range.
1180 These fields include:
1182 - Superblock fields controlled by mount options
1188 - Names present in directory entries, extended attribute keys, and filesystem
1190 - Extended attribute key namespaces
1191 - Extended attribute values
1192 - File data block contents
1194 - Quota timer expiration (if resource usage exceeds the soft limit)
1196 Cross-Referencing Space Metadata
1197 ````````````````````````````````
1199 After internal block checks, the next higher level of checking is
1200 cross-referencing records between metadata structures.
1201 For regular runtime code, the cost of these checks is considered to be
1202 prohibitively expensive, but as scrub is dedicated to rooting out
1203 inconsistencies, it must pursue all avenues of inquiry.
1204 The exact set of cross-referencing is highly dependent on the context of the
1205 data structure being checked.
1207 The XFS btree code has keyspace scanning functions that online fsck uses to
1208 cross reference one structure with another.
1209 Specifically, scrub can scan the key space of an index to determine if that
1210 keyspace is fully, sparsely, or not at all mapped to records.
1211 For the reverse mapping btree, it is possible to mask parts of the key for the
1212 purposes of performing a keyspace scan so that scrub can decide if the rmap
1213 btree contains records mapping a certain extent of physical space without the
1214 sparsenses of the rest of the rmap keyspace getting in the way.
1216 Btree blocks undergo the following checks before cross-referencing:
1218 - Does the type of data stored in the block match what scrub is expecting?
1220 - Does the block belong to the owning structure that asked for the read?
1222 - Do the records fit within the block?
1224 - Are the records contained inside the block free of obvious corruptions?
1226 - Are the name hashes in the correct order?
1228 - Do node pointers within the btree point to valid block addresses for the type
1231 - Do child pointers point towards the leaves?
1233 - Do sibling pointers point across the same level?
1235 - For each node block record, does the record key accurate reflect the contents
1238 Space allocation records are cross-referenced as follows:
1240 1. Any space mentioned by any metadata structure are cross-referenced as
1243 - Does the reverse mapping index list only the appropriate owner as the
1244 owner of each block?
1246 - Are none of the blocks claimed as free space?
1248 - If these aren't file data blocks, are none of the blocks claimed as space
1249 shared by different owners?
1251 2. Btree blocks are cross-referenced as follows:
1253 - Everything in class 1 above.
1255 - If there's a parent node block, do the keys listed for this block match the
1256 keyspace of this block?
1258 - Do the sibling pointers point to valid blocks? Of the same level?
1260 - Do the child pointers point to valid blocks? Of the next level down?
1262 3. Free space btree records are cross-referenced as follows:
1264 - Everything in class 1 and 2 above.
1266 - Does the reverse mapping index list no owners of this space?
1268 - Is this space not claimed by the inode index for inodes?
1270 - Is it not mentioned by the reference count index?
1272 - Is there a matching record in the other free space btree?
1274 4. Inode btree records are cross-referenced as follows:
1276 - Everything in class 1 and 2 above.
1278 - Is there a matching record in free inode btree?
1280 - Do cleared bits in the holemask correspond with inode clusters?
1282 - Do set bits in the freemask correspond with inode records with zero link
1285 5. Inode records are cross-referenced as follows:
1287 - Everything in class 1.
1289 - Do all the fields that summarize information about the file forks actually
1292 - Does each inode with zero link count correspond to a record in the free
1295 6. File fork space mapping records are cross-referenced as follows:
1297 - Everything in class 1 and 2 above.
1299 - Is this space not mentioned by the inode btrees?
1301 - If this is a CoW fork mapping, does it correspond to a CoW entry in the
1302 reference count btree?
1304 7. Reference count records are cross-referenced as follows:
1306 - Everything in class 1 and 2 above.
1308 - Within the space subkeyspace of the rmap btree (that is to say, all
1309 records mapped to a particular space extent and ignoring the owner info),
1310 are there the same number of reverse mapping records for each block as the
1311 reference count record claims?
1313 Proposed patchsets are the series to find gaps in
1315 <https://git.kernel.org/pub/scm/linux/kernel/git/djwong/xfs-linux.git/log/?h=scrub-detect-refcount-gaps>`_,
1317 <https://git.kernel.org/pub/scm/linux/kernel/git/djwong/xfs-linux.git/log/?h=scrub-detect-inobt-gaps>`_, and
1319 <https://git.kernel.org/pub/scm/linux/kernel/git/djwong/xfs-linux.git/log/?h=scrub-detect-rmapbt-gaps>`_ records;
1322 <https://git.kernel.org/pub/scm/linux/kernel/git/djwong/xfs-linux.git/log/?h=scrub-detect-mergeable-records>`_;
1324 `improve cross referencing with rmap
1325 <https://git.kernel.org/pub/scm/linux/kernel/git/djwong/xfs-linux.git/log/?h=scrub-strengthen-rmap-checking>`_
1326 before starting a repair.
1328 Checking Extended Attributes
1329 ````````````````````````````
1331 Extended attributes implement a key-value store that enable fragments of data
1332 to be attached to any file.
1333 Both the kernel and userspace can access the keys and values, subject to
1334 namespace and privilege restrictions.
1335 Most typically these fragments are metadata about the file -- origins, security
1336 contexts, user-supplied labels, indexing information, etc.
1338 Names can be as long as 255 bytes and can exist in several different
1340 Values can be as large as 64KB.
1341 A file's extended attributes are stored in blocks mapped by the attr fork.
1342 The mappings point to leaf blocks, remote value blocks, or dabtree blocks.
1343 Block 0 in the attribute fork is always the top of the structure, but otherwise
1344 each of the three types of blocks can be found at any offset in the attr fork.
1345 Leaf blocks contain attribute key records that point to the name and the value.
1346 Names are always stored elsewhere in the same leaf block.
1347 Values that are less than 3/4 the size of a filesystem block are also stored
1348 elsewhere in the same leaf block.
1349 Remote value blocks contain values that are too large to fit inside a leaf.
1350 If the leaf information exceeds a single filesystem block, a dabtree (also
1351 rooted at block 0) is created to map hashes of the attribute names to leaf
1352 blocks in the attr fork.
1354 Checking an extended attribute structure is not so straightforward due to the
1355 lack of separation between attr blocks and index blocks.
1356 Scrub must read each block mapped by the attr fork and ignore the non-leaf
1359 1. Walk the dabtree in the attr fork (if present) to ensure that there are no
1360 irregularities in the blocks or dabtree mappings that do not point to
1363 2. Walk the blocks of the attr fork looking for leaf blocks.
1364 For each entry inside a leaf:
1366 a. Validate that the name does not contain invalid characters.
1368 b. Read the attr value.
1369 This performs a named lookup of the attr name to ensure the correctness
1371 If the value is stored in a remote block, this also validates the
1372 integrity of the remote value block.
1374 Checking and Cross-Referencing Directories
1375 ``````````````````````````````````````````
1377 The filesystem directory tree is a directed acylic graph structure, with files
1378 constituting the nodes, and directory entries (dirents) constituting the edges.
1379 Directories are a special type of file containing a set of mappings from a
1380 255-byte sequence (name) to an inumber.
1381 These are called directory entries, or dirents for short.
1382 Each directory file must have exactly one directory pointing to the file.
1383 A root directory points to itself.
1384 Directory entries point to files of any type.
1385 Each non-directory file may have multiple directories point to it.
1387 In XFS, directories are implemented as a file containing up to three 32GB
1389 The first partition contains directory entry data blocks.
1390 Each data block contains variable-sized records associating a user-provided
1391 name with an inumber and, optionally, a file type.
1392 If the directory entry data grows beyond one block, the second partition (which
1393 exists as post-EOF extents) is populated with a block containing free space
1394 information and an index that maps hashes of the dirent names to directory data
1395 blocks in the first partition.
1396 This makes directory name lookups very fast.
1397 If this second partition grows beyond one block, the third partition is
1398 populated with a linear array of free space information for faster
1400 If the free space has been separated and the second partition grows again
1401 beyond one block, then a dabtree is used to map hashes of dirent names to
1402 directory data blocks.
1404 Checking a directory is pretty straightforward:
1406 1. Walk the dabtree in the second partition (if present) to ensure that there
1407 are no irregularities in the blocks or dabtree mappings that do not point to
1410 2. Walk the blocks of the first partition looking for directory entries.
1411 Each dirent is checked as follows:
1413 a. Does the name contain no invalid characters?
1415 b. Does the inumber correspond to an actual, allocated inode?
1417 c. Does the child inode have a nonzero link count?
1419 d. If a file type is included in the dirent, does it match the type of the
1422 e. If the child is a subdirectory, does the child's dotdot pointer point
1425 f. If the directory has a second partition, perform a named lookup of the
1426 dirent name to ensure the correctness of the dabtree.
1428 3. Walk the free space list in the third partition (if present) to ensure that
1429 the free spaces it describes are really unused.
1431 Checking operations involving :ref:`parents <dirparent>` and
1432 :ref:`file link counts <nlinks>` are discussed in more detail in later
1435 Checking Directory/Attribute Btrees
1436 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
1438 As stated in previous sections, the directory/attribute btree (dabtree) index
1439 maps user-provided names to improve lookup times by avoiding linear scans.
1440 Internally, it maps a 32-bit hash of the name to a block offset within the
1441 appropriate file fork.
1443 The internal structure of a dabtree closely resembles the btrees that record
1444 fixed-size metadata records -- each dabtree block contains a magic number, a
1445 checksum, sibling pointers, a UUID, a tree level, and a log sequence number.
1446 The format of leaf and node records are the same -- each entry points to the
1447 next level down in the hierarchy, with dabtree node records pointing to dabtree
1448 leaf blocks, and dabtree leaf records pointing to non-dabtree blocks elsewhere
1451 Checking and cross-referencing the dabtree is very similar to what is done for
1454 - Does the type of data stored in the block match what scrub is expecting?
1456 - Does the block belong to the owning structure that asked for the read?
1458 - Do the records fit within the block?
1460 - Are the records contained inside the block free of obvious corruptions?
1462 - Are the name hashes in the correct order?
1464 - Do node pointers within the dabtree point to valid fork offsets for dabtree
1467 - Do leaf pointers within the dabtree point to valid fork offsets for directory
1468 or attr leaf blocks?
1470 - Do child pointers point towards the leaves?
1472 - Do sibling pointers point across the same level?
1474 - For each dabtree node record, does the record key accurate reflect the
1475 contents of the child dabtree block?
1477 - For each dabtree leaf record, does the record key accurate reflect the
1478 contents of the directory or attr block?
1480 Cross-Referencing Summary Counters
1481 ``````````````````````````````````
1483 XFS maintains three classes of summary counters: available resources, quota
1484 resource usage, and file link counts.
1486 In theory, the amount of available resources (data blocks, inodes, realtime
1487 extents) can be found by walking the entire filesystem.
1488 This would make for very slow reporting, so a transactional filesystem can
1489 maintain summaries of this information in the superblock.
1490 Cross-referencing these values against the filesystem metadata should be a
1491 simple matter of walking the free space and inode metadata in each AG and the
1492 realtime bitmap, but there are complications that will be discussed in
1493 :ref:`more detail <fscounters>` later.
1495 :ref:`Quota usage <quotacheck>` and :ref:`file link count <nlinks>`
1496 checking are sufficiently complicated to warrant separate sections.
1498 Post-Repair Reverification
1499 ``````````````````````````
1501 After performing a repair, the checking code is run a second time to validate
1502 the new structure, and the results of the health assessment are recorded
1503 internally and returned to the calling process.
1504 This step is critical for enabling system administrator to monitor the status
1505 of the filesystem and the progress of any repairs.
1506 For developers, it is a useful means to judge the efficacy of error detection
1507 and correction in the online and offline checking tools.
1509 Eventual Consistency vs. Online Fsck
1510 ------------------------------------
1512 Complex operations can make modifications to multiple per-AG data structures
1513 with a chain of transactions.
1514 These chains, once committed to the log, are restarted during log recovery if
1515 the system crashes while processing the chain.
1516 Because the AG header buffers are unlocked between transactions within a chain,
1517 online checking must coordinate with chained operations that are in progress to
1518 avoid incorrectly detecting inconsistencies due to pending chains.
1519 Furthermore, online repair must not run when operations are pending because
1520 the metadata are temporarily inconsistent with each other, and rebuilding is
1523 Only online fsck has this requirement of total consistency of AG metadata, and
1524 should be relatively rare as compared to filesystem change operations.
1525 Online fsck coordinates with transaction chains as follows:
1527 * For each AG, maintain a count of intent items targeting that AG.
1528 The count should be bumped whenever a new item is added to the chain.
1529 The count should be dropped when the filesystem has locked the AG header
1530 buffers and finished the work.
1532 * When online fsck wants to examine an AG, it should lock the AG header
1533 buffers to quiesce all transaction chains that want to modify that AG.
1534 If the count is zero, proceed with the checking operation.
1535 If it is nonzero, cycle the buffer locks to allow the chain to make forward
1538 This may lead to online fsck taking a long time to complete, but regular
1539 filesystem updates take precedence over background checking activity.
1540 Details about the discovery of this situation are presented in the
1541 :ref:`next section <chain_coordination>`, and details about the solution
1542 are presented :ref:`after that<intent_drains>`.
1544 .. _chain_coordination:
1546 Discovery of the Problem
1547 ````````````````````````
1549 Midway through the development of online scrubbing, the fsstress tests
1550 uncovered a misinteraction between online fsck and compound transaction chains
1551 created by other writer threads that resulted in false reports of metadata
1553 The root cause of these reports is the eventual consistency model introduced by
1554 the expansion of deferred work items and compound transaction chains when
1555 reverse mapping and reflink were introduced.
1557 Originally, transaction chains were added to XFS to avoid deadlocks when
1558 unmapping space from files.
1559 Deadlock avoidance rules require that AGs only be locked in increasing order,
1560 which makes it impossible (say) to use a single transaction to free a space
1561 extent in AG 7 and then try to free a now superfluous block mapping btree block
1563 To avoid these kinds of deadlocks, XFS creates Extent Freeing Intent (EFI) log
1564 items to commit to freeing some space in one transaction while deferring the
1565 actual metadata updates to a fresh transaction.
1566 The transaction sequence looks like this:
1568 1. The first transaction contains a physical update to the file's block mapping
1569 structures to remove the mapping from the btree blocks.
1570 It then attaches to the in-memory transaction an action item to schedule
1571 deferred freeing of space.
1572 Concretely, each transaction maintains a list of ``struct
1573 xfs_defer_pending`` objects, each of which maintains a list of ``struct
1574 xfs_extent_free_item`` objects.
1575 Returning to the example above, the action item tracks the freeing of both
1576 the unmapped space from AG 7 and the block mapping btree (BMBT) block from
1578 Deferred frees recorded in this manner are committed in the log by creating
1579 an EFI log item from the ``struct xfs_extent_free_item`` object and
1580 attaching the log item to the transaction.
1581 When the log is persisted to disk, the EFI item is written into the ondisk
1583 EFIs can list up to 16 extents to free, all sorted in AG order.
1585 2. The second transaction contains a physical update to the free space btrees
1586 of AG 3 to release the former BMBT block and a second physical update to the
1587 free space btrees of AG 7 to release the unmapped file space.
1588 Observe that the physical updates are resequenced in the correct order
1590 Attached to the transaction is a an extent free done (EFD) log item.
1591 The EFD contains a pointer to the EFI logged in transaction #1 so that log
1592 recovery can tell if the EFI needs to be replayed.
1594 If the system goes down after transaction #1 is written back to the filesystem
1595 but before #2 is committed, a scan of the filesystem metadata would show
1596 inconsistent filesystem metadata because there would not appear to be any owner
1597 of the unmapped space.
1598 Happily, log recovery corrects this inconsistency for us -- when recovery finds
1599 an intent log item but does not find a corresponding intent done item, it will
1600 reconstruct the incore state of the intent item and finish it.
1601 In the example above, the log must replay both frees described in the recovered
1602 EFI to complete the recovery phase.
1604 There are subtleties to XFS' transaction chaining strategy to consider:
1606 * Log items must be added to a transaction in the correct order to prevent
1607 conflicts with principal objects that are not held by the transaction.
1608 In other words, all per-AG metadata updates for an unmapped block must be
1609 completed before the last update to free the extent, and extents should not
1610 be reallocated until that last update commits to the log.
1612 * AG header buffers are released between each transaction in a chain.
1613 This means that other threads can observe an AG in an intermediate state,
1614 but as long as the first subtlety is handled, this should not affect the
1615 correctness of filesystem operations.
1617 * Unmounting the filesystem flushes all pending work to disk, which means that
1618 offline fsck never sees the temporary inconsistencies caused by deferred
1619 work item processing.
1621 In this manner, XFS employs a form of eventual consistency to avoid deadlocks
1622 and increase parallelism.
1624 During the design phase of the reverse mapping and reflink features, it was
1625 decided that it was impractical to cram all the reverse mapping updates for a
1626 single filesystem change into a single transaction because a single file
1627 mapping operation can explode into many small updates:
1629 * The block mapping update itself
1630 * A reverse mapping update for the block mapping update
1631 * Fixing the freelist
1632 * A reverse mapping update for the freelist fix
1634 * A shape change to the block mapping btree
1635 * A reverse mapping update for the btree update
1636 * Fixing the freelist (again)
1637 * A reverse mapping update for the freelist fix
1639 * An update to the reference counting information
1640 * A reverse mapping update for the refcount update
1641 * Fixing the freelist (a third time)
1642 * A reverse mapping update for the freelist fix
1644 * Freeing any space that was unmapped and not owned by any other file
1645 * Fixing the freelist (a fourth time)
1646 * A reverse mapping update for the freelist fix
1648 * Freeing the space used by the block mapping btree
1649 * Fixing the freelist (a fifth time)
1650 * A reverse mapping update for the freelist fix
1652 Free list fixups are not usually needed more than once per AG per transaction
1653 chain, but it is theoretically possible if space is very tight.
1654 For copy-on-write updates this is even worse, because this must be done once to
1655 remove the space from a staging area and again to map it into the file!
1657 To deal with this explosion in a calm manner, XFS expands its use of deferred
1658 work items to cover most reverse mapping updates and all refcount updates.
1659 This reduces the worst case size of transaction reservations by breaking the
1660 work into a long chain of small updates, which increases the degree of eventual
1661 consistency in the system.
1662 Again, this generally isn't a problem because XFS orders its deferred work
1663 items carefully to avoid resource reuse conflicts between unsuspecting threads.
1665 However, online fsck changes the rules -- remember that although physical
1666 updates to per-AG structures are coordinated by locking the buffers for AG
1667 headers, buffer locks are dropped between transactions.
1668 Once scrub acquires resources and takes locks for a data structure, it must do
1669 all the validation work without releasing the lock.
1670 If the main lock for a space btree is an AG header buffer lock, scrub may have
1671 interrupted another thread that is midway through finishing a chain.
1672 For example, if a thread performing a copy-on-write has completed a reverse
1673 mapping update but not the corresponding refcount update, the two AG btrees
1674 will appear inconsistent to scrub and an observation of corruption will be
1675 recorded. This observation will not be correct.
1676 If a repair is attempted in this state, the results will be catastrophic!
1678 Several other solutions to this problem were evaluated upon discovery of this
1681 1. Add a higher level lock to allocation groups and require writer threads to
1682 acquire the higher level lock in AG order before making any changes.
1683 This would be very difficult to implement in practice because it is
1684 difficult to determine which locks need to be obtained, and in what order,
1685 without simulating the entire operation.
1686 Performing a dry run of a file operation to discover necessary locks would
1687 make the filesystem very slow.
1689 2. Make the deferred work coordinator code aware of consecutive intent items
1690 targeting the same AG and have it hold the AG header buffers locked across
1691 the transaction roll between updates.
1692 This would introduce a lot of complexity into the coordinator since it is
1693 only loosely coupled with the actual deferred work items.
1694 It would also fail to solve the problem because deferred work items can
1695 generate new deferred subtasks, but all subtasks must be complete before
1696 work can start on a new sibling task.
1698 3. Teach online fsck to walk all transactions waiting for whichever lock(s)
1699 protect the data structure being scrubbed to look for pending operations.
1700 The checking and repair operations must factor these pending operations into
1701 the evaluations being performed.
1702 This solution is a nonstarter because it is *extremely* invasive to the main
1710 Online fsck uses an atomic intent item counter and lock cycling to coordinate
1711 with transaction chains.
1712 There are two key properties to the drain mechanism.
1713 First, the counter is incremented when a deferred work item is *queued* to a
1714 transaction, and it is decremented after the associated intent done log item is
1715 *committed* to another transaction.
1716 The second property is that deferred work can be added to a transaction without
1717 holding an AG header lock, but per-AG work items cannot be marked done without
1718 locking that AG header buffer to log the physical updates and the intent done
1720 The first property enables scrub to yield to running transaction chains, which
1721 is an explicit deprioritization of online fsck to benefit file operations.
1722 The second property of the drain is key to the correct coordination of scrub,
1723 since scrub will always be able to decide if a conflict is possible.
1725 For regular filesystem code, the drain works as follows:
1727 1. Call the appropriate subsystem function to add a deferred work item to a
1730 2. The function calls ``xfs_defer_drain_bump`` to increase the counter.
1732 3. When the deferred item manager wants to finish the deferred work item, it
1733 calls ``->finish_item`` to complete it.
1735 4. The ``->finish_item`` implementation logs some changes and calls
1736 ``xfs_defer_drain_drop`` to decrease the sloppy counter and wake up any threads
1737 waiting on the drain.
1739 5. The subtransaction commits, which unlocks the resource associated with the
1742 For scrub, the drain works as follows:
1744 1. Lock the resource(s) associated with the metadata being scrubbed.
1745 For example, a scan of the refcount btree would lock the AGI and AGF header
1748 2. If the counter is zero (``xfs_defer_drain_busy`` returns false), there are no
1749 chains in progress and the operation may proceed.
1751 3. Otherwise, release the resources grabbed in step 1.
1753 4. Wait for the intent counter to reach zero (``xfs_defer_drain_intents``), then go
1754 back to step 1 unless a signal has been caught.
1756 To avoid polling in step 4, the drain provides a waitqueue for scrub threads to
1757 be woken up whenever the intent count drops to zero.
1759 The proposed patchset is the
1760 `scrub intent drain series
1761 <https://git.kernel.org/pub/scm/linux/kernel/git/djwong/xfs-linux.git/log/?h=scrub-drain-intents>`_.
1765 Static Keys (aka Jump Label Patching)
1766 `````````````````````````````````````
1768 Online fsck for XFS separates the regular filesystem from the checking and
1769 repair code as much as possible.
1770 However, there are a few parts of online fsck (such as the intent drains, and
1771 later, live update hooks) where it is useful for the online fsck code to know
1772 what's going on in the rest of the filesystem.
1773 Since it is not expected that online fsck will be constantly running in the
1774 background, it is very important to minimize the runtime overhead imposed by
1775 these hooks when online fsck is compiled into the kernel but not actively
1776 running on behalf of userspace.
1777 Taking locks in the hot path of a writer thread to access a data structure only
1778 to find that no further action is necessary is expensive -- on the author's
1779 computer, this have an overhead of 40-50ns per access.
1780 Fortunately, the kernel supports dynamic code patching, which enables XFS to
1781 replace a static branch to hook code with ``nop`` sleds when online fsck isn't
1783 This sled has an overhead of however long it takes the instruction decoder to
1784 skip past the sled, which seems to be on the order of less than 1ns and
1785 does not access memory outside of instruction fetching.
1787 When online fsck enables the static key, the sled is replaced with an
1788 unconditional branch to call the hook code.
1789 The switchover is quite expensive (~22000ns) but is paid entirely by the
1790 program that invoked online fsck, and can be amortized if multiple threads
1791 enter online fsck at the same time, or if multiple filesystems are being
1792 checked at the same time.
1793 Changing the branch direction requires taking the CPU hotplug lock, and since
1794 CPU initialization requires memory allocation, online fsck must be careful not
1795 to change a static key while holding any locks or resources that could be
1796 accessed in the memory reclaim paths.
1797 To minimize contention on the CPU hotplug lock, care should be taken not to
1798 enable or disable static keys unnecessarily.
1800 Because static keys are intended to minimize hook overhead for regular
1801 filesystem operations when xfs_scrub is not running, the intended usage
1802 patterns are as follows:
1804 - The hooked part of XFS should declare a static-scoped static key that
1806 The ``DEFINE_STATIC_KEY_FALSE`` macro takes care of this.
1807 The static key itself should be declared as a ``static`` variable.
1809 - When deciding to invoke code that's only used by scrub, the regular
1810 filesystem should call the ``static_branch_unlikely`` predicate to avoid the
1811 scrub-only hook code if the static key is not enabled.
1813 - The regular filesystem should export helper functions that call
1814 ``static_branch_inc`` to enable and ``static_branch_dec`` to disable the
1816 Wrapper functions make it easy to compile out the relevant code if the kernel
1817 distributor turns off online fsck at build time.
1819 - Scrub functions wanting to turn on scrub-only XFS functionality should call
1820 the ``xchk_fsgates_enable`` from the setup function to enable a specific
1822 This must be done before obtaining any resources that are used by memory
1824 Callers had better be sure they really need the functionality gated by the
1825 static key; the ``TRY_HARDER`` flag is useful here.
1827 Online scrub has resource acquisition helpers (e.g. ``xchk_perag_lock``) to
1828 handle locking AGI and AGF buffers for all scrubber functions.
1829 If it detects a conflict between scrub and the running transactions, it will
1830 try to wait for intents to complete.
1831 If the caller of the helper has not enabled the static key, the helper will
1832 return -EDEADLOCK, which should result in the scrub being restarted with the
1833 ``TRY_HARDER`` flag set.
1834 The scrub setup function should detect that flag, enable the static key, and
1835 try the scrub again.
1836 Scrub teardown disables all static keys obtained by ``xchk_fsgates_enable``.
1838 For more information, please see the kernel documentation of
1839 Documentation/staging/static-keys.rst.
1843 Pageable Kernel Memory
1844 ----------------------
1846 Some online checking functions work by scanning the filesystem to build a
1847 shadow copy of an ondisk metadata structure in memory and comparing the two
1849 For online repair to rebuild a metadata structure, it must compute the record
1850 set that will be stored in the new structure before it can persist that new
1852 Ideally, repairs complete with a single atomic commit that introduces
1853 a new data structure.
1854 To meet these goals, the kernel needs to collect a large amount of information
1855 in a place that doesn't require the correct operation of the filesystem.
1857 Kernel memory isn't suitable because:
1859 * Allocating a contiguous region of memory to create a C array is very
1860 difficult, especially on 32-bit systems.
1862 * Linked lists of records introduce double pointer overhead which is very high
1863 and eliminate the possibility of indexed lookups.
1865 * Kernel memory is pinned, which can drive the system into OOM conditions.
1867 * The system might not have sufficient memory to stage all the information.
1869 At any given time, online fsck does not need to keep the entire record set in
1870 memory, which means that individual records can be paged out if necessary.
1871 Continued development of online fsck demonstrated that the ability to perform
1872 indexed data storage would also be very useful.
1873 Fortunately, the Linux kernel already has a facility for byte-addressable and
1874 pageable storage: tmpfs.
1875 In-kernel graphics drivers (most notably i915) take advantage of tmpfs files
1876 to store intermediate data that doesn't need to be in memory at all times, so
1877 that usage precedent is already established.
1878 Hence, the ``xfile`` was born!
1880 +--------------------------------------------------------------------------+
1881 | **Historical Sidebar**: |
1882 +--------------------------------------------------------------------------+
1883 | The first edition of online repair inserted records into a new btree as |
1884 | it found them, which failed because filesystem could shut down with a |
1885 | built data structure, which would be live after recovery finished. |
1887 | The second edition solved the half-rebuilt structure problem by storing |
1888 | everything in memory, but frequently ran the system out of memory. |
1890 | The third edition solved the OOM problem by using linked lists, but the |
1891 | memory overhead of the list pointers was extreme. |
1892 +--------------------------------------------------------------------------+
1897 A survey of the intended uses of xfiles suggested these use cases:
1899 1. Arrays of fixed-sized records (space management btrees, directory and
1900 extended attribute entries)
1902 2. Sparse arrays of fixed-sized records (quotas and link counts)
1904 3. Large binary objects (BLOBs) of variable sizes (directory and extended
1905 attribute names and values)
1907 4. Staging btrees in memory (reverse mapping btrees)
1909 5. Arbitrary contents (realtime space management)
1911 To support the first four use cases, high level data structures wrap the xfile
1912 to share functionality between online fsck functions.
1913 The rest of this section discusses the interfaces that the xfile presents to
1914 four of those five higher level data structures.
1915 The fifth use case is discussed in the :ref:`realtime summary <rtsummary>` case
1918 The most general storage interface supported by the xfile enables the reading
1919 and writing of arbitrary quantities of data at arbitrary offsets in the xfile.
1920 This capability is provided by ``xfile_pread`` and ``xfile_pwrite`` functions,
1921 which behave similarly to their userspace counterparts.
1922 XFS is very record-based, which suggests that the ability to load and store
1923 complete records is important.
1924 To support these cases, a pair of ``xfile_obj_load`` and ``xfile_obj_store``
1925 functions are provided to read and persist objects into an xfile.
1926 They are internally the same as pread and pwrite, except that they treat any
1927 error as an out of memory error.
1928 For online repair, squashing error conditions in this manner is an acceptable
1929 behavior because the only reaction is to abort the operation back to userspace.
1930 All five xfile usecases can be serviced by these four functions.
1932 However, no discussion of file access idioms is complete without answering the
1933 question, "But what about mmap?"
1934 It is convenient to access storage directly with pointers, just like userspace
1935 code does with regular memory.
1936 Online fsck must not drive the system into OOM conditions, which means that
1937 xfiles must be responsive to memory reclamation.
1938 tmpfs can only push a pagecache folio to the swap cache if the folio is neither
1939 pinned nor locked, which means the xfile must not pin too many folios.
1941 Short term direct access to xfile contents is done by locking the pagecache
1942 folio and mapping it into kernel address space.
1943 Programmatic access (e.g. pread and pwrite) uses this mechanism.
1944 Folio locks are not supposed to be held for long periods of time, so long
1945 term direct access to xfile contents is done by bumping the folio refcount,
1946 mapping it into kernel address space, and dropping the folio lock.
1947 These long term users *must* be responsive to memory reclaim by hooking into
1948 the shrinker infrastructure to know when to release folios.
1950 The ``xfile_get_page`` and ``xfile_put_page`` functions are provided to
1951 retrieve the (locked) folio that backs part of an xfile and to release it.
1952 The only code to use these folio lease functions are the xfarray
1953 :ref:`sorting<xfarray_sort>` algorithms and the :ref:`in-memory
1956 xfile Access Coordination
1957 `````````````````````````
1959 For security reasons, xfiles must be owned privately by the kernel.
1960 They are marked ``S_PRIVATE`` to prevent interference from the security system,
1961 must never be mapped into process file descriptor tables, and their pages must
1962 never be mapped into userspace processes.
1964 To avoid locking recursion issues with the VFS, all accesses to the shmfs file
1965 are performed by manipulating the page cache directly.
1966 xfile writers call the ``->write_begin`` and ``->write_end`` functions of the
1967 xfile's address space to grab writable pages, copy the caller's buffer into the
1968 page, and release the pages.
1969 xfile readers call ``shmem_read_mapping_page_gfp`` to grab pages directly
1970 before copying the contents into the caller's buffer.
1971 In other words, xfiles ignore the VFS read and write code paths to avoid
1972 having to create a dummy ``struct kiocb`` and to avoid taking inode and
1974 tmpfs cannot be frozen, and xfiles must not be exposed to userspace.
1976 If an xfile is shared between threads to stage repairs, the caller must provide
1977 its own locks to coordinate access.
1978 For example, if a scrub function stores scan results in an xfile and needs
1979 other threads to provide updates to the scanned data, the scrub function must
1980 provide a lock for all threads to share.
1984 Arrays of Fixed-Sized Records
1985 `````````````````````````````
1987 In XFS, each type of indexed space metadata (free space, inodes, reference
1988 counts, file fork space, and reverse mappings) consists of a set of fixed-size
1989 records indexed with a classic B+ tree.
1990 Directories have a set of fixed-size dirent records that point to the names,
1991 and extended attributes have a set of fixed-size attribute keys that point to
1993 Quota counters and file link counters index records with numbers.
1994 During a repair, scrub needs to stage new records during the gathering step and
1995 retrieve them during the btree building step.
1997 Although this requirement can be satisfied by calling the read and write
1998 methods of the xfile directly, it is simpler for callers for there to be a
1999 higher level abstraction to take care of computing array offsets, to provide
2000 iterator functions, and to deal with sparse records and sorting.
2001 The ``xfarray`` abstraction presents a linear array for fixed-size records atop
2002 the byte-accessible xfile.
2004 .. _xfarray_access_patterns:
2006 Array Access Patterns
2007 ^^^^^^^^^^^^^^^^^^^^^
2009 Array access patterns in online fsck tend to fall into three categories.
2010 Iteration of records is assumed to be necessary for all cases and will be
2011 covered in the next section.
2013 The first type of caller handles records that are indexed by position.
2014 Gaps may exist between records, and a record may be updated multiple times
2015 during the collection step.
2016 In other words, these callers want a sparse linearly addressed table file.
2017 The typical use case are quota records or file link count records.
2018 Access to array elements is performed programmatically via ``xfarray_load`` and
2019 ``xfarray_store`` functions, which wrap the similarly-named xfile functions to
2020 provide loading and storing of array elements at arbitrary array indices.
2021 Gaps are defined to be null records, and null records are defined to be a
2022 sequence of all zero bytes.
2023 Null records are detected by calling ``xfarray_element_is_null``.
2024 They are created either by calling ``xfarray_unset`` to null out an existing
2025 record or by never storing anything to an array index.
2027 The second type of caller handles records that are not indexed by position
2028 and do not require multiple updates to a record.
2029 The typical use case here is rebuilding space btrees and key/value btrees.
2030 These callers can add records to the array without caring about array indices
2031 via the ``xfarray_append`` function, which stores a record at the end of the
2033 For callers that require records to be presentable in a specific order (e.g.
2034 rebuilding btree data), the ``xfarray_sort`` function can arrange the sorted
2035 records; this function will be covered later.
2037 The third type of caller is a bag, which is useful for counting records.
2038 The typical use case here is constructing space extent reference counts from
2039 reverse mapping information.
2040 Records can be put in the bag in any order, they can be removed from the bag
2041 at any time, and uniqueness of records is left to callers.
2042 The ``xfarray_store_anywhere`` function is used to insert a record in any
2043 null record slot in the bag; and the ``xfarray_unset`` function removes a
2044 record from the bag.
2046 The proposed patchset is the
2047 `big in-memory array
2048 <https://git.kernel.org/pub/scm/linux/kernel/git/djwong/xfs-linux.git/log/?h=big-array>`_.
2050 Iterating Array Elements
2051 ^^^^^^^^^^^^^^^^^^^^^^^^
2053 Most users of the xfarray require the ability to iterate the records stored in
2055 Callers can probe every possible array index with the following:
2060 foreach_xfarray_idx(array, i) {
2061 xfarray_load(array, i, &rec);
2063 /* do something with rec */
2066 All users of this idiom must be prepared to handle null records or must already
2067 know that there aren't any.
2069 For xfarray users that want to iterate a sparse array, the ``xfarray_iter``
2070 function ignores indices in the xfarray that have never been written to by
2071 calling ``xfile_seek_data`` (which internally uses ``SEEK_DATA``) to skip areas
2072 of the array that are not populated with memory pages.
2073 Once it finds a page, it will skip the zeroed areas of the page.
2077 xfarray_idx_t i = XFARRAY_CURSOR_INIT;
2078 while ((ret = xfarray_iter(array, &i, &rec)) == 1) {
2079 /* do something with rec */
2084 Sorting Array Elements
2085 ^^^^^^^^^^^^^^^^^^^^^^
2087 During the fourth demonstration of online repair, a community reviewer remarked
2088 that for performance reasons, online repair ought to load batches of records
2089 into btree record blocks instead of inserting records into a new btree one at a
2091 The btree insertion code in XFS is responsible for maintaining correct ordering
2092 of the records, so naturally the xfarray must also support sorting the record
2093 set prior to bulk loading.
2095 Case Study: Sorting xfarrays
2096 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~
2098 The sorting algorithm used in the xfarray is actually a combination of adaptive
2099 quicksort and a heapsort subalgorithm in the spirit of
2100 `Sedgewick <https://algs4.cs.princeton.edu/23quicksort/>`_ and
2101 `pdqsort <https://github.com/orlp/pdqsort>`_, with customizations for the Linux
2103 To sort records in a reasonably short amount of time, ``xfarray`` takes
2104 advantage of the binary subpartitioning offered by quicksort, but it also uses
2105 heapsort to hedge against performance collapse if the chosen quicksort pivots
2107 Both algorithms are (in general) O(n * lg(n)), but there is a wide performance
2108 gulf between the two implementations.
2110 The Linux kernel already contains a reasonably fast implementation of heapsort.
2111 It only operates on regular C arrays, which limits the scope of its usefulness.
2112 There are two key places where the xfarray uses it:
2114 * Sorting any record subset backed by a single xfile page.
2116 * Loading a small number of xfarray records from potentially disparate parts
2117 of the xfarray into a memory buffer, and sorting the buffer.
2119 In other words, ``xfarray`` uses heapsort to constrain the nested recursion of
2120 quicksort, thereby mitigating quicksort's worst runtime behavior.
2122 Choosing a quicksort pivot is a tricky business.
2123 A good pivot splits the set to sort in half, leading to the divide and conquer
2124 behavior that is crucial to O(n * lg(n)) performance.
2125 A poor pivot barely splits the subset at all, leading to O(n\ :sup:`2`)
2127 The xfarray sort routine tries to avoid picking a bad pivot by sampling nine
2128 records into a memory buffer and using the kernel heapsort to identify the
2131 Most modern quicksort implementations employ Tukey's "ninther" to select a
2132 pivot from a classic C array.
2133 Typical ninther implementations pick three unique triads of records, sort each
2134 of the triads, and then sort the middle value of each triad to determine the
2136 As stated previously, however, xfile accesses are not entirely cheap.
2137 It turned out to be much more performant to read the nine elements into a
2138 memory buffer, run the kernel's in-memory heapsort on the buffer, and choose
2139 the 4th element of that buffer as the pivot.
2140 Tukey's ninthers are described in J. W. Tukey, `The ninther, a technique for
2141 low-effort robust (resistant) location in large samples`, in *Contributions to
2142 Survey Sampling and Applied Statistics*, edited by H. David, (Academic Press,
2145 The partitioning of quicksort is fairly textbook -- rearrange the record
2146 subset around the pivot, then set up the current and next stack frames to
2147 sort with the larger and the smaller halves of the pivot, respectively.
2148 This keeps the stack space requirements to log2(record count).
2150 As a final performance optimization, the hi and lo scanning phase of quicksort
2151 keeps examined xfile pages mapped in the kernel for as long as possible to
2152 reduce map/unmap cycles.
2153 Surprisingly, this reduces overall sort runtime by nearly half again after
2154 accounting for the application of heapsort directly onto xfile pages.
2161 Extended attributes and directories add an additional requirement for staging
2162 records: arbitrary byte sequences of finite length.
2163 Each directory entry record needs to store entry name,
2164 and each extended attribute needs to store both the attribute name and value.
2165 The names, keys, and values can consume a large amount of memory, so the
2166 ``xfblob`` abstraction was created to simplify management of these blobs
2169 Blob arrays provide ``xfblob_load`` and ``xfblob_store`` functions to retrieve
2170 and persist objects.
2171 The store function returns a magic cookie for every object that it persists.
2172 Later, callers provide this cookie to the ``xblob_load`` to recall the object.
2173 The ``xfblob_free`` function frees a specific blob, and the ``xfblob_truncate``
2174 function frees them all because compaction is not needed.
2176 The details of repairing directories and extended attributes will be discussed
2177 in a subsequent section about atomic extent swapping.
2178 However, it should be noted that these repair functions only use blob storage
2179 to cache a small number of entries before adding them to a temporary ondisk
2180 file, which is why compaction is not required.
2182 The proposed patchset is at the start of the
2183 `extended attribute repair
2184 <https://git.kernel.org/pub/scm/linux/kernel/git/djwong/xfs-linux.git/log/?h=repair-xattrs>`_ series.
2191 The chapter about :ref:`secondary metadata<secondary_metadata>` mentioned that
2192 checking and repairing of secondary metadata commonly requires coordination
2193 between a live metadata scan of the filesystem and writer threads that are
2194 updating that metadata.
2195 Keeping the scan data up to date requires requires the ability to propagate
2196 metadata updates from the filesystem into the data being collected by the scan.
2197 This *can* be done by appending concurrent updates into a separate log file and
2198 applying them before writing the new metadata to disk, but this leads to
2199 unbounded memory consumption if the rest of the system is very busy.
2200 Another option is to skip the side-log and commit live updates from the
2201 filesystem directly into the scan data, which trades more overhead for a lower
2202 maximum memory requirement.
2203 In both cases, the data structure holding the scan results must support indexed
2204 access to perform well.
2206 Given that indexed lookups of scan data is required for both strategies, online
2207 fsck employs the second strategy of committing live updates directly into
2209 Because xfarrays are not indexed and do not enforce record ordering, they
2210 are not suitable for this task.
2211 Conveniently, however, XFS has a library to create and maintain ordered reverse
2212 mapping records: the existing rmap btree code!
2213 If only there was a means to create one in memory.
2215 Recall that the :ref:`xfile <xfile>` abstraction represents memory pages as a
2216 regular file, which means that the kernel can create byte or block addressable
2217 virtual address spaces at will.
2218 The XFS buffer cache specializes in abstracting IO to block-oriented address
2219 spaces, which means that adaptation of the buffer cache to interface with
2220 xfiles enables reuse of the entire btree library.
2221 Btrees built atop an xfile are collectively known as ``xfbtrees``.
2222 The next few sections describe how they actually work.
2224 The proposed patchset is the
2226 <https://git.kernel.org/pub/scm/linux/kernel/git/djwong/xfs-linux.git/log/?h=in-memory-btrees>`_
2229 Using xfiles as a Buffer Cache Target
2230 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2232 Two modifications are necessary to support xfiles as a buffer cache target.
2233 The first is to make it possible for the ``struct xfs_buftarg`` structure to
2234 host the ``struct xfs_buf`` rhashtable, because normally those are held by a
2236 The second change is to modify the buffer ``ioapply`` function to "read" cached
2237 pages from the xfile and "write" cached pages back to the xfile.
2238 Multiple access to individual buffers is controlled by the ``xfs_buf`` lock,
2239 since the xfile does not provide any locking on its own.
2240 With this adaptation in place, users of the xfile-backed buffer cache use
2241 exactly the same APIs as users of the disk-backed buffer cache.
2242 The separation between xfile and buffer cache implies higher memory usage since
2243 they do not share pages, but this property could some day enable transactional
2244 updates to an in-memory btree.
2245 Today, however, it simply eliminates the need for new code.
2247 Space Management with an xfbtree
2248 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2250 Space management for an xfile is very simple -- each btree block is one memory
2252 These blocks use the same header format as an on-disk btree, but the in-memory
2253 block verifiers ignore the checksums, assuming that xfile memory is no more
2254 corruption-prone than regular DRAM.
2255 Reusing existing code here is more important than absolute memory efficiency.
2257 The very first block of an xfile backing an xfbtree contains a header block.
2258 The header describes the owner, height, and the block number of the root
2261 To allocate a btree block, use ``xfile_seek_data`` to find a gap in the file.
2262 If there are no gaps, create one by extending the length of the xfile.
2263 Preallocate space for the block with ``xfile_prealloc``, and hand back the
2265 To free an xfbtree block, use ``xfile_discard`` (which internally uses
2266 ``FALLOC_FL_PUNCH_HOLE``) to remove the memory page from the xfile.
2268 Populating an xfbtree
2269 ^^^^^^^^^^^^^^^^^^^^^
2271 An online fsck function that wants to create an xfbtree should proceed as
2274 1. Call ``xfile_create`` to create an xfile.
2276 2. Call ``xfs_alloc_memory_buftarg`` to create a buffer cache target structure
2277 pointing to the xfile.
2279 3. Pass the buffer cache target, buffer ops, and other information to
2280 ``xfbtree_create`` to write an initial tree header and root block to the
2282 Each btree type should define a wrapper that passes necessary arguments to
2283 the creation function.
2284 For example, rmap btrees define ``xfs_rmapbt_mem_create`` to take care of
2285 all the necessary details for callers.
2286 A ``struct xfbtree`` object will be returned.
2288 4. Pass the xfbtree object to the btree cursor creation function for the
2290 Following the example above, ``xfs_rmapbt_mem_cursor`` takes care of this
2293 5. Pass the btree cursor to the regular btree functions to make queries against
2294 and to update the in-memory btree.
2295 For example, a btree cursor for an rmap xfbtree can be passed to the
2296 ``xfs_rmap_*`` functions just like any other btree cursor.
2297 See the :ref:`next section<xfbtree_commit>` for information on dealing with
2298 xfbtree updates that are logged to a transaction.
2300 6. When finished, delete the btree cursor, destroy the xfbtree object, free the
2301 buffer target, and the destroy the xfile to release all resources.
2305 Committing Logged xfbtree Buffers
2306 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2308 Although it is a clever hack to reuse the rmap btree code to handle the staging
2309 structure, the ephemeral nature of the in-memory btree block storage presents
2310 some challenges of its own.
2311 The XFS transaction manager must not commit buffer log items for buffers backed
2312 by an xfile because the log format does not understand updates for devices
2313 other than the data device.
2314 An ephemeral xfbtree probably will not exist by the time the AIL checkpoints
2315 log transactions back into the filesystem, and certainly won't exist during
2317 For these reasons, any code updating an xfbtree in transaction context must
2318 remove the buffer log items from the transaction and write the updates into the
2319 backing xfile before committing or cancelling the transaction.
2321 The ``xfbtree_trans_commit`` and ``xfbtree_trans_cancel`` functions implement
2322 this functionality as follows:
2324 1. Find each buffer log item whose buffer targets the xfile.
2326 2. Record the dirty/ordered status of the log item.
2328 3. Detach the log item from the buffer.
2330 4. Queue the buffer to a special delwri list.
2332 5. Clear the transaction dirty flag if the only dirty log items were the ones
2333 that were detached in step 3.
2335 6. Submit the delwri list to commit the changes to the xfile, if the updates
2336 are being committed.
2338 After removing xfile logged buffers from the transaction in this manner, the
2339 transaction can be committed or cancelled.
2341 Bulk Loading of Ondisk B+Trees
2342 ------------------------------
2344 As mentioned previously, early iterations of online repair built new btree
2345 structures by creating a new btree and adding observations individually.
2346 Loading a btree one record at a time had a slight advantage of not requiring
2347 the incore records to be sorted prior to commit, but was very slow and leaked
2348 blocks if the system went down during a repair.
2349 Loading records one at a time also meant that repair could not control the
2350 loading factor of the blocks in the new btree.
2352 Fortunately, the venerable ``xfs_repair`` tool had a more efficient means for
2353 rebuilding a btree index from a collection of records -- bulk btree loading.
2354 This was implemented rather inefficiently code-wise, since ``xfs_repair``
2355 had separate copy-pasted implementations for each btree type.
2357 To prepare for online fsck, each of the four bulk loaders were studied, notes
2358 were taken, and the four were refactored into a single generic btree bulk
2360 Those notes in turn have been refreshed and are presented below.
2362 Geometry Computation
2363 ````````````````````
2365 The zeroth step of bulk loading is to assemble the entire record set that will
2366 be stored in the new btree, and sort the records.
2367 Next, call ``xfs_btree_bload_compute_geometry`` to compute the shape of the
2368 btree from the record set, the type of btree, and any load factor preferences.
2369 This information is required for resource reservation.
2371 First, the geometry computation computes the minimum and maximum records that
2372 will fit in a leaf block from the size of a btree block and the size of the
2374 Roughly speaking, the maximum number of records is::
2376 maxrecs = (block_size - header_size) / record_size
2378 The XFS design specifies that btree blocks should be merged when possible,
2379 which means the minimum number of records is half of maxrecs::
2381 minrecs = maxrecs / 2
2383 The next variable to determine is the desired loading factor.
2384 This must be at least minrecs and no more than maxrecs.
2385 Choosing minrecs is undesirable because it wastes half the block.
2386 Choosing maxrecs is also undesirable because adding a single record to each
2387 newly rebuilt leaf block will cause a tree split, which causes a noticeable
2388 drop in performance immediately afterwards.
2389 The default loading factor was chosen to be 75% of maxrecs, which provides a
2390 reasonably compact structure without any immediate split penalties::
2392 default_load_factor = (maxrecs + minrecs) / 2
2394 If space is tight, the loading factor will be set to maxrecs to try to avoid
2395 running out of space::
2397 leaf_load_factor = enough space ? default_load_factor : maxrecs
2399 Load factor is computed for btree node blocks using the combined size of the
2400 btree key and pointer as the record size::
2402 maxrecs = (block_size - header_size) / (key_size + ptr_size)
2403 minrecs = maxrecs / 2
2404 node_load_factor = enough space ? default_load_factor : maxrecs
2406 Once that's done, the number of leaf blocks required to store the record set
2407 can be computed as::
2409 leaf_blocks = ceil(record_count / leaf_load_factor)
2411 The number of node blocks needed to point to the next level down in the tree
2414 n_blocks = (n == 0 ? leaf_blocks : node_blocks[n])
2415 node_blocks[n + 1] = ceil(n_blocks / node_load_factor)
2417 The entire computation is performed recursively until the current level only
2419 The resulting geometry is as follows:
2421 - For AG-rooted btrees, this level is the root level, so the height of the new
2422 tree is ``level + 1`` and the space needed is the summation of the number of
2423 blocks on each level.
2425 - For inode-rooted btrees where the records in the top level do not fit in the
2426 inode fork area, the height is ``level + 2``, the space needed is the
2427 summation of the number of blocks on each level, and the inode fork points to
2430 - For inode-rooted btrees where the records in the top level can be stored in
2431 the inode fork area, then the root block can be stored in the inode, the
2432 height is ``level + 1``, and the space needed is one less than the summation
2433 of the number of blocks on each level.
2434 This only becomes relevant when non-bmap btrees gain the ability to root in
2435 an inode, which is a future patchset and only included here for completeness.
2439 Reserving New B+Tree Blocks
2440 ```````````````````````````
2442 Once repair knows the number of blocks needed for the new btree, it allocates
2443 those blocks using the free space information.
2444 Each reserved extent is tracked separately by the btree builder state data.
2445 To improve crash resilience, the reservation code also logs an Extent Freeing
2446 Intent (EFI) item in the same transaction as each space allocation and attaches
2447 its in-memory ``struct xfs_extent_free_item`` object to the space reservation.
2448 If the system goes down, log recovery will use the unfinished EFIs to free the
2449 unused space, the free space, leaving the filesystem unchanged.
2451 Each time the btree builder claims a block for the btree from a reserved
2452 extent, it updates the in-memory reservation to reflect the claimed space.
2453 Block reservation tries to allocate as much contiguous space as possible to
2454 reduce the number of EFIs in play.
2456 While repair is writing these new btree blocks, the EFIs created for the space
2457 reservations pin the tail of the ondisk log.
2458 It's possible that other parts of the system will remain busy and push the head
2459 of the log towards the pinned tail.
2460 To avoid livelocking the filesystem, the EFIs must not pin the tail of the log
2462 To alleviate this problem, the dynamic relogging capability of the deferred ops
2463 mechanism is reused here to commit a transaction at the log head containing an
2464 EFD for the old EFI and new EFI at the head.
2465 This enables the log to release the old EFI to keep the log moving forwards.
2467 EFIs have a role to play during the commit and reaping phases; please see the
2468 next section and the section about :ref:`reaping<reaping>` for more details.
2470 Proposed patchsets are the
2472 <https://git.kernel.org/pub/scm/linux/kernel/git/djwong/xfs-linux.git/log/?h=repair-bitmap-rework>`_
2474 `preparation for bulk loading btrees
2475 <https://git.kernel.org/pub/scm/linux/kernel/git/djwong/xfs-linux.git/log/?h=repair-prep-for-bulk-loading>`_.
2478 Writing the New Tree
2479 ````````````````````
2481 This part is pretty simple -- the btree builder (``xfs_btree_bulkload``) claims
2482 a block from the reserved list, writes the new btree block header, fills the
2483 rest of the block with records, and adds the new leaf block to a list of
2491 Sibling pointers are set every time a new block is added to the level::
2493 ┌────┐ ┌────┐ ┌────┐ ┌────┐
2494 │leaf│→│leaf│→│leaf│→│leaf│
2495 │RRR │←│RRR │←│RRR │←│RRR │
2496 └────┘ └────┘ └────┘ └────┘
2498 When it finishes writing the record leaf blocks, it moves on to the node
2500 To fill a node block, it walks each block in the next level down in the tree
2501 to compute the relevant keys and write them into the parent node::
2508 ┌────┐ ┌────┐ ┌────┐ ┌────┐
2509 │leaf│→│leaf│→│leaf│→│leaf│
2510 │RRR │←│RRR │←│RRR │←│RRR │
2511 └────┘ └────┘ └────┘ └────┘
2513 When it reaches the root level, it is ready to commit the new btree!::
2525 ┌────┐ ┌────┐ ┌────┐ ┌────┐
2526 │leaf│→│leaf│→│leaf│→│leaf│
2527 │RRR │←│RRR │←│RRR │←│RRR │
2528 └────┘ └────┘ └────┘ └────┘
2530 The first step to commit the new btree is to persist the btree blocks to disk
2532 This is a little complicated because a new btree block could have been freed
2533 in the recent past, so the builder must use ``xfs_buf_delwri_queue_here`` to
2534 remove the (stale) buffer from the AIL list before it can write the new blocks
2536 Blocks are queued for IO using a delwri list and written in one large batch
2537 with ``xfs_buf_delwri_submit``.
2539 Once the new blocks have been persisted to disk, control returns to the
2540 individual repair function that called the bulk loader.
2541 The repair function must log the location of the new root in a transaction,
2542 clean up the space reservations that were made for the new btree, and reap the
2543 old metadata blocks:
2545 1. Commit the location of the new btree root.
2547 2. For each incore reservation:
2549 a. Log Extent Freeing Done (EFD) items for all the space that was consumed
2550 by the btree builder. The new EFDs must point to the EFIs attached to
2551 the reservation to prevent log recovery from freeing the new blocks.
2553 b. For unclaimed portions of incore reservations, create a regular deferred
2554 extent free work item to be free the unused space later in the
2557 c. The EFDs and EFIs logged in steps 2a and 2b must not overrun the
2558 reservation of the committing transaction.
2559 If the btree loading code suspects this might be about to happen, it must
2560 call ``xrep_defer_finish`` to clear out the deferred work and obtain a
2563 3. Clear out the deferred work a second time to finish the commit and clean
2564 the repair transaction.
2566 The transaction rolling in steps 2c and 3 represent a weakness in the repair
2567 algorithm, because a log flush and a crash before the end of the reap step can
2568 result in space leaking.
2569 Online repair functions minimize the chances of this occurring by using very
2570 large transactions, which each can accommodate many thousands of block freeing
2572 Repair moves on to reaping the old blocks, which will be presented in a
2573 subsequent :ref:`section<reaping>` after a few case studies of bulk loading.
2575 Case Study: Rebuilding the Inode Index
2576 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2578 The high level process to rebuild the inode index btree is:
2580 1. Walk the reverse mapping records to generate ``struct xfs_inobt_rec``
2581 records from the inode chunk information and a bitmap of the old inode btree
2584 2. Append the records to an xfarray in inode order.
2586 3. Use the ``xfs_btree_bload_compute_geometry`` function to compute the number
2587 of blocks needed for the inode btree.
2588 If the free space inode btree is enabled, call it again to estimate the
2589 geometry of the finobt.
2591 4. Allocate the number of blocks computed in the previous step.
2593 5. Use ``xfs_btree_bload`` to write the xfarray records to btree blocks and
2594 generate the internal node blocks.
2595 If the free space inode btree is enabled, call it again to load the finobt.
2597 6. Commit the location of the new btree root block(s) to the AGI.
2599 7. Reap the old btree blocks using the bitmap created in step 1.
2601 Details are as follows.
2603 The inode btree maps inumbers to the ondisk location of the associated
2604 inode records, which means that the inode btrees can be rebuilt from the
2605 reverse mapping information.
2606 Reverse mapping records with an owner of ``XFS_RMAP_OWN_INOBT`` marks the
2607 location of the old inode btree blocks.
2608 Each reverse mapping record with an owner of ``XFS_RMAP_OWN_INODES`` marks the
2609 location of at least one inode cluster buffer.
2610 A cluster is the smallest number of ondisk inodes that can be allocated or
2611 freed in a single transaction; it is never smaller than 1 fs block or 4 inodes.
2613 For the space represented by each inode cluster, ensure that there are no
2614 records in the free space btrees nor any records in the reference count btree.
2615 If there are, the space metadata inconsistencies are reason enough to abort the
2617 Otherwise, read each cluster buffer to check that its contents appear to be
2618 ondisk inodes and to decide if the file is allocated
2619 (``xfs_dinode.i_mode != 0``) or free (``xfs_dinode.i_mode == 0``).
2620 Accumulate the results of successive inode cluster buffer reads until there is
2621 enough information to fill a single inode chunk record, which is 64 consecutive
2622 numbers in the inumber keyspace.
2623 If the chunk is sparse, the chunk record may include holes.
2625 Once the repair function accumulates one chunk's worth of data, it calls
2626 ``xfarray_append`` to add the inode btree record to the xfarray.
2627 This xfarray is walked twice during the btree creation step -- once to populate
2628 the inode btree with all inode chunk records, and a second time to populate the
2629 free inode btree with records for chunks that have free non-sparse inodes.
2630 The number of records for the inode btree is the number of xfarray records,
2631 but the record count for the free inode btree has to be computed as inode chunk
2632 records are stored in the xfarray.
2634 The proposed patchset is the
2636 <https://git.kernel.org/pub/scm/linux/kernel/git/djwong/xfs-linux.git/log/?h=repair-ag-btrees>`_
2639 Case Study: Rebuilding the Space Reference Counts
2640 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2642 Reverse mapping records are used to rebuild the reference count information.
2643 Reference counts are required for correct operation of copy on write for shared
2645 Imagine the reverse mapping entries as rectangles representing extents of
2646 physical blocks, and that the rectangles can be laid down to allow them to
2648 From the diagram below, it is apparent that a reference count record must start
2649 or end wherever the height of the stack changes.
2650 In other words, the record emission stimulus is level-triggered::
2653 ██ █████ ████ ███ ██████
2654 ██ ████ ███████████ ████ █████████
2655 ████████████████████████████████ ███████████
2656 ^ ^ ^^ ^^ ^ ^^ ^^^ ^^^^ ^ ^^ ^ ^ ^
2657 2 1 23 21 3 43 234 2123 1 01 2 3 0
2659 The ondisk reference count btree does not store the refcount == 0 cases because
2660 the free space btree already records which blocks are free.
2661 Extents being used to stage copy-on-write operations should be the only records
2663 Single-owner file blocks aren't recorded in either the free space or the
2664 reference count btrees.
2666 The high level process to rebuild the reference count btree is:
2668 1. Walk the reverse mapping records to generate ``struct xfs_refcount_irec``
2669 records for any space having more than one reverse mapping and add them to
2671 Any records owned by ``XFS_RMAP_OWN_COW`` are also added to the xfarray
2672 because these are extents allocated to stage a copy on write operation and
2673 are tracked in the refcount btree.
2675 Use any records owned by ``XFS_RMAP_OWN_REFC`` to create a bitmap of old
2676 refcount btree blocks.
2678 2. Sort the records in physical extent order, putting the CoW staging extents
2679 at the end of the xfarray.
2680 This matches the sorting order of records in the refcount btree.
2682 3. Use the ``xfs_btree_bload_compute_geometry`` function to compute the number
2683 of blocks needed for the new tree.
2685 4. Allocate the number of blocks computed in the previous step.
2687 5. Use ``xfs_btree_bload`` to write the xfarray records to btree blocks and
2688 generate the internal node blocks.
2690 6. Commit the location of new btree root block to the AGF.
2692 7. Reap the old btree blocks using the bitmap created in step 1.
2694 Details are as follows; the same algorithm is used by ``xfs_repair`` to
2695 generate refcount information from reverse mapping records.
2697 - Until the reverse mapping btree runs out of records:
2699 - Retrieve the next record from the btree and put it in a bag.
2701 - Collect all records with the same starting block from the btree and put
2704 - While the bag isn't empty:
2706 - Among the mappings in the bag, compute the lowest block number where the
2707 reference count changes.
2708 This position will be either the starting block number of the next
2709 unprocessed reverse mapping or the next block after the shortest mapping
2712 - Remove all mappings from the bag that end at this position.
2714 - Collect all reverse mappings that start at this position from the btree
2715 and put them in the bag.
2717 - If the size of the bag changed and is greater than one, create a new
2718 refcount record associating the block number range that we just walked to
2719 the size of the bag.
2721 The bag-like structure in this case is a type 2 xfarray as discussed in the
2722 :ref:`xfarray access patterns<xfarray_access_patterns>` section.
2723 Reverse mappings are added to the bag using ``xfarray_store_anywhere`` and
2724 removed via ``xfarray_unset``.
2725 Bag members are examined through ``xfarray_iter`` loops.
2727 The proposed patchset is the
2729 <https://git.kernel.org/pub/scm/linux/kernel/git/djwong/xfs-linux.git/log/?h=repair-ag-btrees>`_
2732 Case Study: Rebuilding File Fork Mapping Indices
2733 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2735 The high level process to rebuild a data/attr fork mapping btree is:
2737 1. Walk the reverse mapping records to generate ``struct xfs_bmbt_rec``
2738 records from the reverse mapping records for that inode and fork.
2739 Append these records to an xfarray.
2740 Compute the bitmap of the old bmap btree blocks from the ``BMBT_BLOCK``
2743 2. Use the ``xfs_btree_bload_compute_geometry`` function to compute the number
2744 of blocks needed for the new tree.
2746 3. Sort the records in file offset order.
2748 4. If the extent records would fit in the inode fork immediate area, commit the
2749 records to that immediate area and skip to step 8.
2751 5. Allocate the number of blocks computed in the previous step.
2753 6. Use ``xfs_btree_bload`` to write the xfarray records to btree blocks and
2754 generate the internal node blocks.
2756 7. Commit the new btree root block to the inode fork immediate area.
2758 8. Reap the old btree blocks using the bitmap created in step 1.
2760 There are some complications here:
2761 First, it's possible to move the fork offset to adjust the sizes of the
2762 immediate areas if the data and attr forks are not both in BMBT format.
2763 Second, if there are sufficiently few fork mappings, it may be possible to use
2764 EXTENTS format instead of BMBT, which may require a conversion.
2765 Third, the incore extent map must be reloaded carefully to avoid disturbing
2766 any delayed allocation extents.
2768 The proposed patchset is the
2769 `file mapping repair
2770 <https://git.kernel.org/pub/scm/linux/kernel/git/djwong/xfs-linux.git/log/?h=repair-file-mappings>`_
2775 Reaping Old Metadata Blocks
2776 ---------------------------
2778 Whenever online fsck builds a new data structure to replace one that is
2779 suspect, there is a question of how to find and dispose of the blocks that
2780 belonged to the old structure.
2781 The laziest method of course is not to deal with them at all, but this slowly
2782 leads to service degradations as space leaks out of the filesystem.
2783 Hopefully, someone will schedule a rebuild of the free space information to
2784 plug all those leaks.
2785 Offline repair rebuilds all space metadata after recording the usage of
2786 the files and directories that it decides not to clear, hence it can build new
2787 structures in the discovered free space and avoid the question of reaping.
2789 As part of a repair, online fsck relies heavily on the reverse mapping records
2790 to find space that is owned by the corresponding rmap owner yet truly free.
2791 Cross referencing rmap records with other rmap records is necessary because
2792 there may be other data structures that also think they own some of those
2793 blocks (e.g. crosslinked trees).
2794 Permitting the block allocator to hand them out again will not push the system
2795 towards consistency.
2797 For space metadata, the process of finding extents to dispose of generally
2798 follows this format:
2800 1. Create a bitmap of space used by data structures that must be preserved.
2801 The space reservations used to create the new metadata can be used here if
2802 the same rmap owner code is used to denote all of the objects being rebuilt.
2804 2. Survey the reverse mapping data to create a bitmap of space owned by the
2805 same ``XFS_RMAP_OWN_*`` number for the metadata that is being preserved.
2807 3. Use the bitmap disunion operator to subtract (1) from (2).
2808 The remaining set bits represent candidate extents that could be freed.
2809 The process moves on to step 4 below.
2811 Repairs for file-based metadata such as extended attributes, directories,
2812 symbolic links, quota files and realtime bitmaps are performed by building a
2813 new structure attached to a temporary file and swapping the forks.
2814 Afterward, the mappings in the old file fork are the candidate blocks for
2817 The process for disposing of old extents is as follows:
2819 4. For each candidate extent, count the number of reverse mapping records for
2820 the first block in that extent that do not have the same rmap owner for the
2821 data structure being repaired.
2823 - If zero, the block has a single owner and can be freed.
2825 - If not, the block is part of a crosslinked structure and must not be
2828 5. Starting with the next block in the extent, figure out how many more blocks
2829 have the same zero/nonzero other owner status as that first block.
2831 6. If the region is crosslinked, delete the reverse mapping entry for the
2832 structure being repaired and move on to the next region.
2834 7. If the region is to be freed, mark any corresponding buffers in the buffer
2835 cache as stale to prevent log writeback.
2837 8. Free the region and move on.
2839 However, there is one complication to this procedure.
2840 Transactions are of finite size, so the reaping process must be careful to roll
2841 the transactions to avoid overruns.
2842 Overruns come from two sources:
2844 a. EFIs logged on behalf of space that is no longer occupied
2846 b. Log items for buffer invalidations
2848 This is also a window in which a crash during the reaping process can leak
2850 As stated earlier, online repair functions use very large transactions to
2851 minimize the chances of this occurring.
2853 The proposed patchset is the
2854 `preparation for bulk loading btrees
2855 <https://git.kernel.org/pub/scm/linux/kernel/git/djwong/xfs-linux.git/log/?h=repair-prep-for-bulk-loading>`_
2858 Case Study: Reaping After a Regular Btree Repair
2859 ````````````````````````````````````````````````
2861 Old reference count and inode btrees are the easiest to reap because they have
2862 rmap records with special owner codes: ``XFS_RMAP_OWN_REFC`` for the refcount
2863 btree, and ``XFS_RMAP_OWN_INOBT`` for the inode and free inode btrees.
2864 Creating a list of extents to reap the old btree blocks is quite simple,
2867 1. Lock the relevant AGI/AGF header buffers to prevent allocation and frees.
2869 2. For each reverse mapping record with an rmap owner corresponding to the
2870 metadata structure being rebuilt, set the corresponding range in a bitmap.
2872 3. Walk the current data structures that have the same rmap owner.
2873 For each block visited, clear that range in the above bitmap.
2875 4. Each set bit in the bitmap represents a block that could be a block from the
2876 old data structures and hence is a candidate for reaping.
2877 In other words, ``(rmap_records_owned_by & ~blocks_reachable_by_walk)``
2878 are the blocks that might be freeable.
2880 If it is possible to maintain the AGF lock throughout the repair (which is the
2881 common case), then step 2 can be performed at the same time as the reverse
2882 mapping record walk that creates the records for the new btree.
2884 Case Study: Rebuilding the Free Space Indices
2885 `````````````````````````````````````````````
2887 The high level process to rebuild the free space indices is:
2889 1. Walk the reverse mapping records to generate ``struct xfs_alloc_rec_incore``
2890 records from the gaps in the reverse mapping btree.
2892 2. Append the records to an xfarray.
2894 3. Use the ``xfs_btree_bload_compute_geometry`` function to compute the number
2895 of blocks needed for each new tree.
2897 4. Allocate the number of blocks computed in the previous step from the free
2898 space information collected.
2900 5. Use ``xfs_btree_bload`` to write the xfarray records to btree blocks and
2901 generate the internal node blocks for the free space by length index.
2902 Call it again for the free space by block number index.
2904 6. Commit the locations of the new btree root blocks to the AGF.
2906 7. Reap the old btree blocks by looking for space that is not recorded by the
2907 reverse mapping btree, the new free space btrees, or the AGFL.
2909 Repairing the free space btrees has three key complications over a regular
2912 First, free space is not explicitly tracked in the reverse mapping records.
2913 Hence, the new free space records must be inferred from gaps in the physical
2914 space component of the keyspace of the reverse mapping btree.
2916 Second, free space repairs cannot use the common btree reservation code because
2917 new blocks are reserved out of the free space btrees.
2918 This is impossible when repairing the free space btrees themselves.
2919 However, repair holds the AGF buffer lock for the duration of the free space
2920 index reconstruction, so it can use the collected free space information to
2921 supply the blocks for the new free space btrees.
2922 It is not necessary to back each reserved extent with an EFI because the new
2923 free space btrees are constructed in what the ondisk filesystem thinks is
2925 However, if reserving blocks for the new btrees from the collected free space
2926 information changes the number of free space records, repair must re-estimate
2927 the new free space btree geometry with the new record count until the
2928 reservation is sufficient.
2929 As part of committing the new btrees, repair must ensure that reverse mappings
2930 are created for the reserved blocks and that unused reserved blocks are
2931 inserted into the free space btrees.
2932 Deferrred rmap and freeing operations are used to ensure that this transition
2933 is atomic, similar to the other btree repair functions.
2935 Third, finding the blocks to reap after the repair is not overly
2937 Blocks for the free space btrees and the reverse mapping btrees are supplied by
2939 Blocks put onto the AGFL have reverse mapping records with the owner
2940 ``XFS_RMAP_OWN_AG``.
2941 This ownership is retained when blocks move from the AGFL into the free space
2942 btrees or the reverse mapping btrees.
2943 When repair walks reverse mapping records to synthesize free space records, it
2944 creates a bitmap (``ag_owner_bitmap``) of all the space claimed by
2945 ``XFS_RMAP_OWN_AG`` records.
2946 The repair context maintains a second bitmap corresponding to the rmap btree
2947 blocks and the AGFL blocks (``rmap_agfl_bitmap``).
2948 When the walk is complete, the bitmap disunion operation ``(ag_owner_bitmap &
2949 ~rmap_agfl_bitmap)`` computes the extents that are used by the old free space
2951 These blocks can then be reaped using the methods outlined above.
2953 The proposed patchset is the
2955 <https://git.kernel.org/pub/scm/linux/kernel/git/djwong/xfs-linux.git/log/?h=repair-ag-btrees>`_
2960 Case Study: Reaping After Repairing Reverse Mapping Btrees
2961 ``````````````````````````````````````````````````````````
2963 Old reverse mapping btrees are less difficult to reap after a repair.
2964 As mentioned in the previous section, blocks on the AGFL, the two free space
2965 btree blocks, and the reverse mapping btree blocks all have reverse mapping
2966 records with ``XFS_RMAP_OWN_AG`` as the owner.
2967 The full process of gathering reverse mapping records and building a new btree
2968 are described in the case study of
2969 :ref:`live rebuilds of rmap data <rmap_repair>`, but a crucial point from that
2970 discussion is that the new rmap btree will not contain any records for the old
2971 rmap btree, nor will the old btree blocks be tracked in the free space btrees.
2972 The list of candidate reaping blocks is computed by setting the bits
2973 corresponding to the gaps in the new rmap btree records, and then clearing the
2974 bits corresponding to extents in the free space btrees and the current AGFL
2976 The result ``(new_rmapbt_gaps & ~(agfl | bnobt_records))`` are reaped using the
2977 methods outlined above.
2979 The rest of the process of rebuildng the reverse mapping btree is discussed
2980 in a separate :ref:`case study<rmap_repair>`.
2982 The proposed patchset is the
2984 <https://git.kernel.org/pub/scm/linux/kernel/git/djwong/xfs-linux.git/log/?h=repair-ag-btrees>`_
2987 Case Study: Rebuilding the AGFL
2988 ```````````````````````````````
2990 The allocation group free block list (AGFL) is repaired as follows:
2992 1. Create a bitmap for all the space that the reverse mapping data claims is
2993 owned by ``XFS_RMAP_OWN_AG``.
2995 2. Subtract the space used by the two free space btrees and the rmap btree.
2997 3. Subtract any space that the reverse mapping data claims is owned by any
2998 other owner, to avoid re-adding crosslinked blocks to the AGFL.
3000 4. Once the AGFL is full, reap any blocks leftover.
3002 5. The next operation to fix the freelist will right-size the list.
3004 See `fs/xfs/scrub/agheader_repair.c <https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/fs/xfs/scrub/agheader_repair.c>`_ for more details.
3006 Inode Record Repairs
3007 --------------------
3009 Inode records must be handled carefully, because they have both ondisk records
3010 ("dinodes") and an in-memory ("cached") representation.
3011 There is a very high potential for cache coherency issues if online fsck is not
3012 careful to access the ondisk metadata *only* when the ondisk metadata is so
3013 badly damaged that the filesystem cannot load the in-memory representation.
3014 When online fsck wants to open a damaged file for scrubbing, it must use
3015 specialized resource acquisition functions that return either the in-memory
3016 representation *or* a lock on whichever object is necessary to prevent any
3017 update to the ondisk location.
3019 The only repairs that should be made to the ondisk inode buffers are whatever
3020 is necessary to get the in-core structure loaded.
3021 This means fixing whatever is caught by the inode cluster buffer and inode fork
3022 verifiers, and retrying the ``iget`` operation.
3023 If the second ``iget`` fails, the repair has failed.
3025 Once the in-memory representation is loaded, repair can lock the inode and can
3026 subject it to comprehensive checks, repairs, and optimizations.
3027 Most inode attributes are easy to check and constrain, or are user-controlled
3028 arbitrary bit patterns; these are both easy to fix.
3029 Dealing with the data and attr fork extent counts and the file block counts is
3030 more complicated, because computing the correct value requires traversing the
3031 forks, or if that fails, leaving the fields invalid and waiting for the fork
3032 fsck functions to run.
3034 The proposed patchset is the
3036 <https://git.kernel.org/pub/scm/linux/kernel/git/djwong/xfs-linux.git/log/?h=repair-inodes>`_
3039 Quota Record Repairs
3040 --------------------
3042 Similar to inodes, quota records ("dquots") also have both ondisk records and
3043 an in-memory representation, and hence are subject to the same cache coherency
3045 Somewhat confusingly, both are known as dquots in the XFS codebase.
3047 The only repairs that should be made to the ondisk quota record buffers are
3048 whatever is necessary to get the in-core structure loaded.
3049 Once the in-memory representation is loaded, the only attributes needing
3050 checking are obviously bad limits and timer values.
3052 Quota usage counters are checked, repaired, and discussed separately in the
3053 section about :ref:`live quotacheck <quotacheck>`.
3055 The proposed patchset is the
3057 <https://git.kernel.org/pub/scm/linux/kernel/git/djwong/xfs-linux.git/log/?h=repair-quota>`_
3062 Freezing to Fix Summary Counters
3063 --------------------------------
3065 Filesystem summary counters track availability of filesystem resources such
3066 as free blocks, free inodes, and allocated inodes.
3067 This information could be compiled by walking the free space and inode indexes,
3068 but this is a slow process, so XFS maintains a copy in the ondisk superblock
3069 that should reflect the ondisk metadata, at least when the filesystem has been
3071 For performance reasons, XFS also maintains incore copies of those counters,
3072 which are key to enabling resource reservations for active transactions.
3073 Writer threads reserve the worst-case quantities of resources from the
3074 incore counter and give back whatever they don't use at commit time.
3075 It is therefore only necessary to serialize on the superblock when the
3076 superblock is being committed to disk.
3078 The lazy superblock counter feature introduced in XFS v5 took this even further
3079 by training log recovery to recompute the summary counters from the AG headers,
3080 which eliminated the need for most transactions even to touch the superblock.
3081 The only time XFS commits the summary counters is at filesystem unmount.
3082 To reduce contention even further, the incore counter is implemented as a
3083 percpu counter, which means that each CPU is allocated a batch of blocks from a
3084 global incore counter and can satisfy small allocations from the local batch.
3086 The high-performance nature of the summary counters makes it difficult for
3087 online fsck to check them, since there is no way to quiesce a percpu counter
3088 while the system is running.
3089 Although online fsck can read the filesystem metadata to compute the correct
3090 values of the summary counters, there's no way to hold the value of a percpu
3091 counter stable, so it's quite possible that the counter will be out of date by
3092 the time the walk is complete.
3093 Earlier versions of online scrub would return to userspace with an incomplete
3094 scan flag, but this is not a satisfying outcome for a system administrator.
3095 For repairs, the in-memory counters must be stabilized while walking the
3096 filesystem metadata to get an accurate reading and install it in the percpu
3099 To satisfy this requirement, online fsck must prevent other programs in the
3100 system from initiating new writes to the filesystem, it must disable background
3101 garbage collection threads, and it must wait for existing writer programs to
3103 Once that has been established, scrub can walk the AG free space indexes, the
3104 inode btrees, and the realtime bitmap to compute the correct value of all
3105 four summary counters.
3106 This is very similar to a filesystem freeze, though not all of the pieces are
3109 - The final freeze state is set one higher than ``SB_FREEZE_COMPLETE`` to
3110 prevent other threads from thawing the filesystem, or other scrub threads
3111 from initiating another fscounters freeze.
3113 - It does not quiesce the log.
3115 With this code in place, it is now possible to pause the filesystem for just
3116 long enough to check and correct the summary counters.
3118 +--------------------------------------------------------------------------+
3119 | **Historical Sidebar**: |
3120 +--------------------------------------------------------------------------+
3121 | The initial implementation used the actual VFS filesystem freeze |
3122 | mechanism to quiesce filesystem activity. |
3123 | With the filesystem frozen, it is possible to resolve the counter values |
3124 | with exact precision, but there are many problems with calling the VFS |
3125 | methods directly: |
3127 | - Other programs can unfreeze the filesystem without our knowledge. |
3128 | This leads to incorrect scan results and incorrect repairs. |
3130 | - Adding an extra lock to prevent others from thawing the filesystem |
3131 | required the addition of a ``->freeze_super`` function to wrap |
3132 | ``freeze_fs()``. |
3133 | This in turn caused other subtle problems because it turns out that |
3134 | the VFS ``freeze_super`` and ``thaw_super`` functions can drop the |
3135 | last reference to the VFS superblock, and any subsequent access |
3136 | becomes a UAF bug! |
3137 | This can happen if the filesystem is unmounted while the underlying |
3138 | block device has frozen the filesystem. |
3139 | This problem could be solved by grabbing extra references to the |
3140 | superblock, but it felt suboptimal given the other inadequacies of |
3143 | - The log need not be quiesced to check the summary counters, but a VFS |
3144 | freeze initiates one anyway. |
3145 | This adds unnecessary runtime to live fscounter fsck operations. |
3147 | - Quiescing the log means that XFS flushes the (possibly incorrect) |
3148 | counters to disk as part of cleaning the log. |
3150 | - A bug in the VFS meant that freeze could complete even when |
3151 | sync_filesystem fails to flush the filesystem and returns an error. |
3152 | This bug was fixed in Linux 5.17. |
3153 +--------------------------------------------------------------------------+
3155 The proposed patchset is the
3156 `summary counter cleanup
3157 <https://git.kernel.org/pub/scm/linux/kernel/git/djwong/xfs-linux.git/log/?h=repair-fscounters>`_
3160 Full Filesystem Scans
3161 ---------------------
3163 Certain types of metadata can only be checked by walking every file in the
3164 entire filesystem to record observations and comparing the observations against
3165 what's recorded on disk.
3166 Like every other type of online repair, repairs are made by writing those
3167 observations to disk in a replacement structure and committing it atomically.
3168 However, it is not practical to shut down the entire filesystem to examine
3169 hundreds of billions of files because the downtime would be excessive.
3170 Therefore, online fsck must build the infrastructure to manage a live scan of
3171 all the files in the filesystem.
3172 There are two questions that need to be solved to perform a live walk:
3174 - How does scrub manage the scan while it is collecting data?
3176 - How does the scan keep abreast of changes being made to the system by other
3181 Coordinated Inode Scans
3182 ```````````````````````
3184 In the original Unix filesystems of the 1970s, each directory entry contained
3185 an index number (*inumber*) which was used as an index into on ondisk array
3186 (*itable*) of fixed-size records (*inodes*) describing a file's attributes and
3187 its data block mapping.
3188 This system is described by J. Lions, `"inode (5659)"
3189 <http://www.lemis.com/grog/Documentation/Lions/>`_ in *Lions' Commentary on
3190 UNIX, 6th Edition*, (Dept. of Computer Science, the University of New South
3191 Wales, November 1977), pp. 18-2; and later by D. Ritchie and K. Thompson,
3192 `"Implementation of the File System"
3193 <https://archive.org/details/bstj57-6-1905/page/n8/mode/1up>`_, from *The UNIX
3194 Time-Sharing System*, (The Bell System Technical Journal, July 1978), pp.
3197 XFS retains most of this design, except now inumbers are search keys over all
3198 the space in the data section filesystem.
3199 They form a continuous keyspace that can be expressed as a 64-bit integer,
3200 though the inodes themselves are sparsely distributed within the keyspace.
3201 Scans proceed in a linear fashion across the inumber keyspace, starting from
3202 ``0x0`` and ending at ``0xFFFFFFFFFFFFFFFF``.
3203 Naturally, a scan through a keyspace requires a scan cursor object to track the
3205 Because this keyspace is sparse, this cursor contains two parts.
3206 The first part of this scan cursor object tracks the inode that will be
3207 examined next; call this the examination cursor.
3208 Somewhat less obviously, the scan cursor object must also track which parts of
3209 the keyspace have already been visited, which is critical for deciding if a
3210 concurrent filesystem update needs to be incorporated into the scan data.
3211 Call this the visited inode cursor.
3213 Advancing the scan cursor is a multi-step process encapsulated in
3214 ``xchk_iscan_iter``:
3216 1. Lock the AGI buffer of the AG containing the inode pointed to by the visited
3218 This guarantee that inodes in this AG cannot be allocated or freed while
3219 advancing the cursor.
3221 2. Use the per-AG inode btree to look up the next inumber after the one that
3222 was just visited, since it may not be keyspace adjacent.
3224 3. If there are no more inodes left in this AG:
3226 a. Move the examination cursor to the point of the inumber keyspace that
3227 corresponds to the start of the next AG.
3229 b. Adjust the visited inode cursor to indicate that it has "visited" the
3230 last possible inode in the current AG's inode keyspace.
3231 XFS inumbers are segmented, so the cursor needs to be marked as having
3232 visited the entire keyspace up to just before the start of the next AG's
3235 c. Unlock the AGI and return to step 1 if there are unexamined AGs in the
3238 d. If there are no more AGs to examine, set both cursors to the end of the
3240 The scan is now complete.
3242 4. Otherwise, there is at least one more inode to scan in this AG:
3244 a. Move the examination cursor ahead to the next inode marked as allocated
3247 b. Adjust the visited inode cursor to point to the inode just prior to where
3248 the examination cursor is now.
3249 Because the scanner holds the AGI buffer lock, no inodes could have been
3250 created in the part of the inode keyspace that the visited inode cursor
3253 5. Get the incore inode for the inumber of the examination cursor.
3254 By maintaining the AGI buffer lock until this point, the scanner knows that
3255 it was safe to advance the examination cursor across the entire keyspace,
3256 and that it has stabilized this next inode so that it cannot disappear from
3257 the filesystem until the scan releases the incore inode.
3259 6. Drop the AGI lock and return the incore inode to the caller.
3261 Online fsck functions scan all files in the filesystem as follows:
3263 1. Start a scan by calling ``xchk_iscan_start``.
3265 2. Advance the scan cursor (``xchk_iscan_iter``) to get the next inode.
3268 a. Lock the inode to prevent updates during the scan.
3272 c. While still holding the inode lock, adjust the visited inode cursor
3273 (``xchk_iscan_mark_visited``) to point to this inode.
3275 d. Unlock and release the inode.
3277 8. Call ``xchk_iscan_teardown`` to complete the scan.
3279 There are subtleties with the inode cache that complicate grabbing the incore
3280 inode for the caller.
3281 Obviously, it is an absolute requirement that the inode metadata be consistent
3282 enough to load it into the inode cache.
3283 Second, if the incore inode is stuck in some intermediate state, the scan
3284 coordinator must release the AGI and push the main filesystem to get the inode
3285 back into a loadable state.
3287 The proposed patches are the
3289 <https://git.kernel.org/pub/scm/linux/kernel/git/djwong/xfs-linux.git/log/?h=scrub-iscan>`_
3291 The first user of the new functionality is the
3293 <https://git.kernel.org/pub/scm/linux/kernel/git/djwong/xfs-linux.git/log/?h=repair-quotacheck>`_
3299 In regular filesystem code, references to allocated XFS incore inodes are
3300 always obtained (``xfs_iget``) outside of transaction context because the
3301 creation of the incore context for an existing file does not require metadata
3303 However, it is important to note that references to incore inodes obtained as
3304 part of file creation must be performed in transaction context because the
3305 filesystem must ensure the atomicity of the ondisk inode btree index updates
3306 and the initialization of the actual ondisk inode.
3308 References to incore inodes are always released (``xfs_irele``) outside of
3309 transaction context because there are a handful of activities that might
3310 require ondisk updates:
3312 - The VFS may decide to kick off writeback as part of a ``DONTCACHE`` inode
3315 - Speculative preallocations need to be unreserved.
3317 - An unlinked file may have lost its last reference, in which case the entire
3318 file must be inactivated, which involves releasing all of its resources in
3319 the ondisk metadata and freeing the inode.
3321 These activities are collectively called inode inactivation.
3322 Inactivation has two parts -- the VFS part, which initiates writeback on all
3323 dirty file pages, and the XFS part, which cleans up XFS-specific information
3324 and frees the inode if it was unlinked.
3325 If the inode is unlinked (or unconnected after a file handle operation), the
3326 kernel drops the inode into the inactivation machinery immediately.
3328 During normal operation, resource acquisition for an update follows this order
3331 1. Inode reference (``iget``).
3333 2. Filesystem freeze protection, if repairing (``mnt_want_write_file``).
3335 3. Inode ``IOLOCK`` (VFS ``i_rwsem``) lock to control file IO.
3337 4. Inode ``MMAPLOCK`` (page cache ``invalidate_lock``) lock for operations that
3338 can update page cache mappings.
3340 5. Log feature enablement.
3342 6. Transaction log space grant.
3344 7. Space on the data and realtime devices for the transaction.
3346 8. Incore dquot references, if a file is being repaired.
3347 Note that they are not locked, merely acquired.
3349 9. Inode ``ILOCK`` for file metadata updates.
3351 10. AG header buffer locks / Realtime metadata inode ILOCK.
3353 11. Realtime metadata buffer locks, if applicable.
3355 12. Extent mapping btree blocks, if applicable.
3357 Resources are often released in the reverse order, though this is not required.
3358 However, online fsck differs from regular XFS operations because it may examine
3359 an object that normally is acquired in a later stage of the locking order, and
3360 then decide to cross-reference the object with an object that is acquired
3361 earlier in the order.
3362 The next few sections detail the specific ways in which online fsck takes care
3365 iget and irele During a Scrub
3366 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
3368 An inode scan performed on behalf of a scrub operation runs in transaction
3369 context, and possibly with resources already locked and bound to it.
3370 This isn't much of a problem for ``iget`` since it can operate in the context
3371 of an existing transaction, as long as all of the bound resources are acquired
3372 before the inode reference in the regular filesystem.
3374 When the VFS ``iput`` function is given a linked inode with no other
3375 references, it normally puts the inode on an LRU list in the hope that it can
3376 save time if another process re-opens the file before the system runs out
3377 of memory and frees it.
3378 Filesystem callers can short-circuit the LRU process by setting a ``DONTCACHE``
3379 flag on the inode to cause the kernel to try to drop the inode into the
3380 inactivation machinery immediately.
3382 In the past, inactivation was always done from the process that dropped the
3383 inode, which was a problem for scrub because scrub may already hold a
3384 transaction, and XFS does not support nesting transactions.
3385 On the other hand, if there is no scrub transaction, it is desirable to drop
3386 otherwise unused inodes immediately to avoid polluting caches.
3387 To capture these nuances, the online fsck code has a separate ``xchk_irele``
3388 function to set or clear the ``DONTCACHE`` flag to get the required release
3391 Proposed patchsets include fixing
3393 <https://git.kernel.org/pub/scm/linux/kernel/git/djwong/xfs-linux.git/log/?h=scrub-iget-fixes>`_ and
3395 <https://git.kernel.org/pub/scm/linux/kernel/git/djwong/xfs-linux.git/log/?h=scrub-dir-iget-fixes>`_.
3402 In regular filesystem code, the VFS and XFS will acquire multiple IOLOCK locks
3403 in a well-known order: parent → child when updating the directory tree, and
3404 in numerical order of the addresses of their ``struct inode`` object otherwise.
3405 For regular files, the MMAPLOCK can be acquired after the IOLOCK to stop page
3407 If two MMAPLOCKs must be acquired, they are acquired in numerical order of
3408 the addresses of their ``struct address_space`` objects.
3409 Due to the structure of existing filesystem code, IOLOCKs and MMAPLOCKs must be
3410 acquired before transactions are allocated.
3411 If two ILOCKs must be acquired, they are acquired in inumber order.
3413 Inode lock acquisition must be done carefully during a coordinated inode scan.
3414 Online fsck cannot abide these conventions, because for a directory tree
3415 scanner, the scrub process holds the IOLOCK of the file being scanned and it
3416 needs to take the IOLOCK of the file at the other end of the directory link.
3417 If the directory tree is corrupt because it contains a cycle, ``xfs_scrub``
3418 cannot use the regular inode locking functions and avoid becoming trapped in an
3421 Solving both of these problems is straightforward -- any time online fsck
3422 needs to take a second lock of the same class, it uses trylock to avoid an ABBA
3424 If the trylock fails, scrub drops all inode locks and use trylock loops to
3425 (re)acquire all necessary resources.
3426 Trylock loops enable scrub to check for pending fatal signals, which is how
3427 scrub avoids deadlocking the filesystem or becoming an unresponsive process.
3428 However, trylock loops means that online fsck must be prepared to measure the
3429 resource being scrubbed before and after the lock cycle to detect changes and
3434 Case Study: Finding a Directory Parent
3435 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
3437 Consider the directory parent pointer repair code as an example.
3438 Online fsck must verify that the dotdot dirent of a directory points up to a
3439 parent directory, and that the parent directory contains exactly one dirent
3440 pointing down to the child directory.
3441 Fully validating this relationship (and repairing it if possible) requires a
3442 walk of every directory on the filesystem while holding the child locked, and
3443 while updates to the directory tree are being made.
3444 The coordinated inode scan provides a way to walk the filesystem without the
3445 possibility of missing an inode.
3446 The child directory is kept locked to prevent updates to the dotdot dirent, but
3447 if the scanner fails to lock a parent, it can drop and relock both the child
3448 and the prospective parent.
3449 If the dotdot entry changes while the directory is unlocked, then a move or
3450 rename operation must have changed the child's parentage, and the scan can
3453 The proposed patchset is the
3455 <https://git.kernel.org/pub/scm/linux/kernel/git/djwong/xfs-linux.git/log/?h=repair-dirs>`_
3463 The second piece of support that online fsck functions need during a full
3464 filesystem scan is the ability to stay informed about updates being made by
3465 other threads in the filesystem, since comparisons against the past are useless
3466 in a dynamic environment.
3467 Two pieces of Linux kernel infrastructure enable online fsck to monitor regular
3468 filesystem operations: filesystem hooks and :ref:`static keys<jump_labels>`.
3470 Filesystem hooks convey information about an ongoing filesystem operation to
3471 a downstream consumer.
3472 In this case, the downstream consumer is always an online fsck function.
3473 Because multiple fsck functions can run in parallel, online fsck uses the Linux
3474 notifier call chain facility to dispatch updates to any number of interested
3476 Call chains are a dynamic list, which means that they can be configured at
3478 Because these hooks are private to the XFS module, the information passed along
3479 contains exactly what the checking function needs to update its observations.
3481 The current implementation of XFS hooks uses SRCU notifier chains to reduce the
3482 impact to highly threaded workloads.
3483 Regular blocking notifier chains use a rwsem and seem to have a much lower
3484 overhead for single-threaded applications.
3485 However, it may turn out that the combination of blocking chains and static
3486 keys are a more performant combination; more study is needed here.
3488 The following pieces are necessary to hook a certain point in the filesystem:
3490 - A ``struct xfs_hooks`` object must be embedded in a convenient place such as
3491 a well-known incore filesystem object.
3493 - Each hook must define an action code and a structure containing more context
3496 - Hook providers should provide appropriate wrapper functions and structs
3497 around the ``xfs_hooks`` and ``xfs_hook`` objects to take advantage of type
3498 checking to ensure correct usage.
3500 - A callsite in the regular filesystem code must be chosen to call
3501 ``xfs_hooks_call`` with the action code and data structure.
3502 This place should be adjacent to (and not earlier than) the place where
3503 the filesystem update is committed to the transaction.
3504 In general, when the filesystem calls a hook chain, it should be able to
3505 handle sleeping and should not be vulnerable to memory reclaim or locking
3507 However, the exact requirements are very dependent on the context of the hook
3508 caller and the callee.
3510 - The online fsck function should define a structure to hold scan data, a lock
3511 to coordinate access to the scan data, and a ``struct xfs_hook`` object.
3512 The scanner function and the regular filesystem code must acquire resources
3513 in the same order; see the next section for details.
3515 - The online fsck code must contain a C function to catch the hook action code
3517 If the object being updated has already been visited by the scan, then the
3518 hook information must be applied to the scan data.
3520 - Prior to unlocking inodes to start the scan, online fsck must call
3521 ``xfs_hooks_setup`` to initialize the ``struct xfs_hook``, and
3522 ``xfs_hooks_add`` to enable the hook.
3524 - Online fsck must call ``xfs_hooks_del`` to disable the hook once the scan is
3527 The number of hooks should be kept to a minimum to reduce complexity.
3528 Static keys are used to reduce the overhead of filesystem hooks to nearly
3529 zero when online fsck is not running.
3533 Live Updates During a Scan
3534 ``````````````````````````
3536 The code paths of the online fsck scanning code and the :ref:`hooked<fshooks>`
3537 filesystem code look like this::
3541 inode lock ←────────────────────┐
3545 filesystem function │
3547 notifier call chain │ same
3549 scrub hook function │ lock
3551 scan data mutex ←──┐ same │
3553 update scan data │ lock │
3555 scan data mutex ←──┘ │
3557 inode lock ←────────────────────┘
3565 These rules must be followed to ensure correct interactions between the
3566 checking code and the code making an update to the filesystem:
3568 - Prior to invoking the notifier call chain, the filesystem function being
3569 hooked must acquire the same lock that the scrub scanning function acquires
3572 - The scanning function and the scrub hook function must coordinate access to
3573 the scan data by acquiring a lock on the scan data.
3575 - Scrub hook function must not add the live update information to the scan
3576 observations unless the inode being updated has already been scanned.
3577 The scan coordinator has a helper predicate (``xchk_iscan_want_live_update``)
3580 - Scrub hook functions must not change the caller's state, including the
3581 transaction that it is running.
3582 They must not acquire any resources that might conflict with the filesystem
3583 function being hooked.
3585 - The hook function can abort the inode scan to avoid breaking the other rules.
3587 The inode scan APIs are pretty simple:
3589 - ``xchk_iscan_start`` starts a scan
3591 - ``xchk_iscan_iter`` grabs a reference to the next inode in the scan or
3592 returns zero if there is nothing left to scan
3594 - ``xchk_iscan_want_live_update`` to decide if an inode has already been
3595 visited in the scan.
3596 This is critical for hook functions to decide if they need to update the
3597 in-memory scan information.
3599 - ``xchk_iscan_mark_visited`` to mark an inode as having been visited in the
3602 - ``xchk_iscan_teardown`` to finish the scan
3604 This functionality is also a part of the
3606 <https://git.kernel.org/pub/scm/linux/kernel/git/djwong/xfs-linux.git/log/?h=scrub-iscan>`_
3611 Case Study: Quota Counter Checking
3612 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
3614 It is useful to compare the mount time quotacheck code to the online repair
3616 Mount time quotacheck does not have to contend with concurrent operations, so
3617 it does the following:
3619 1. Make sure the ondisk dquots are in good enough shape that all the incore
3620 dquots will actually load, and zero the resource usage counters in the
3623 2. Walk every inode in the filesystem.
3624 Add each file's resource usage to the incore dquot.
3626 3. Walk each incore dquot.
3627 If the incore dquot is not being flushed, add the ondisk buffer backing the
3628 incore dquot to a delayed write (delwri) list.
3630 4. Write the buffer list to disk.
3632 Like most online fsck functions, online quotacheck can't write to regular
3633 filesystem objects until the newly collected metadata reflect all filesystem
3635 Therefore, online quotacheck records file resource usage to a shadow dquot
3636 index implemented with a sparse ``xfarray``, and only writes to the real dquots
3637 once the scan is complete.
3638 Handling transactional updates is tricky because quota resource usage updates
3639 are handled in phases to minimize contention on dquots:
3641 1. The inodes involved are joined and locked to a transaction.
3643 2. For each dquot attached to the file:
3645 a. The dquot is locked.
3647 b. A quota reservation is added to the dquot's resource usage.
3648 The reservation is recorded in the transaction.
3650 c. The dquot is unlocked.
3652 3. Changes in actual quota usage are tracked in the transaction.
3654 4. At transaction commit time, each dquot is examined again:
3656 a. The dquot is locked again.
3658 b. Quota usage changes are logged and unused reservation is given back to
3661 c. The dquot is unlocked.
3663 For online quotacheck, hooks are placed in steps 2 and 4.
3664 The step 2 hook creates a shadow version of the transaction dquot context
3665 (``dqtrx``) that operates in a similar manner to the regular code.
3666 The step 4 hook commits the shadow ``dqtrx`` changes to the shadow dquots.
3667 Notice that both hooks are called with the inode locked, which is how the
3668 live update coordinates with the inode scanner.
3670 The quotacheck scan looks like this:
3672 1. Set up a coordinated inode scan.
3674 2. For each inode returned by the inode scan iterator:
3676 a. Grab and lock the inode.
3678 b. Determine that inode's resource usage (data blocks, inode counts,
3679 realtime blocks) and add that to the shadow dquots for the user, group,
3680 and project ids associated with the inode.
3682 c. Unlock and release the inode.
3684 3. For each dquot in the system:
3686 a. Grab and lock the dquot.
3688 b. Check the dquot against the shadow dquots created by the scan and updated
3691 Live updates are key to being able to walk every quota record without
3692 needing to hold any locks for a long duration.
3693 If repairs are desired, the real and shadow dquots are locked and their
3694 resource counts are set to the values in the shadow dquot.
3696 The proposed patchset is the
3698 <https://git.kernel.org/pub/scm/linux/kernel/git/djwong/xfs-linux.git/log/?h=repair-quotacheck>`_
3703 Case Study: File Link Count Checking
3704 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
3706 File link count checking also uses live update hooks.
3707 The coordinated inode scanner is used to visit all directories on the
3708 filesystem, and per-file link count records are stored in a sparse ``xfarray``
3710 During the scanning phase, each entry in a directory generates observation
3713 1. If the entry is a dotdot (``'..'``) entry of the root directory, the
3714 directory's parent link count is bumped because the root directory's dotdot
3715 entry is self referential.
3717 2. If the entry is a dotdot entry of a subdirectory, the parent's backref
3720 3. If the entry is neither a dot nor a dotdot entry, the target file's parent
3723 4. If the target is a subdirectory, the parent's child link count is bumped.
3725 A crucial point to understand about how the link count inode scanner interacts
3726 with the live update hooks is that the scan cursor tracks which *parent*
3727 directories have been scanned.
3728 In other words, the live updates ignore any update about ``A → B`` when A has
3729 not been scanned, even if B has been scanned.
3730 Furthermore, a subdirectory A with a dotdot entry pointing back to B is
3731 accounted as a backref counter in the shadow data for A, since child dotdot
3732 entries affect the parent's link count.
3733 Live update hooks are carefully placed in all parts of the filesystem that
3734 create, change, or remove directory entries, since those operations involve
3735 bumplink and droplink.
3737 For any file, the correct link count is the number of parents plus the number
3738 of child subdirectories.
3739 Non-directories never have children of any kind.
3740 The backref information is used to detect inconsistencies in the number of
3741 links pointing to child subdirectories and the number of dotdot entries
3744 After the scan completes, the link count of each file can be checked by locking
3745 both the inode and the shadow data, and comparing the link counts.
3746 A second coordinated inode scan cursor is used for comparisons.
3747 Live updates are key to being able to walk every inode without needing to hold
3748 any locks between inodes.
3749 If repairs are desired, the inode's link count is set to the value in the
3751 If no parents are found, the file must be :ref:`reparented <orphanage>` to the
3752 orphanage to prevent the file from being lost forever.
3754 The proposed patchset is the
3755 `file link count repair
3756 <https://git.kernel.org/pub/scm/linux/kernel/git/djwong/xfs-linux.git/log/?h=scrub-nlinks>`_
3761 Case Study: Rebuilding Reverse Mapping Records
3762 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
3764 Most repair functions follow the same pattern: lock filesystem resources,
3765 walk the surviving ondisk metadata looking for replacement metadata records,
3766 and use an :ref:`in-memory array <xfarray>` to store the gathered observations.
3767 The primary advantage of this approach is the simplicity and modularity of the
3768 repair code -- code and data are entirely contained within the scrub module,
3769 do not require hooks in the main filesystem, and are usually the most efficient
3771 A secondary advantage of this repair approach is atomicity -- once the kernel
3772 decides a structure is corrupt, no other threads can access the metadata until
3773 the kernel finishes repairing and revalidating the metadata.
3775 For repairs going on within a shard of the filesystem, these advantages
3776 outweigh the delays inherent in locking the shard while repairing parts of the
3778 Unfortunately, repairs to the reverse mapping btree cannot use the "standard"
3779 btree repair strategy because it must scan every space mapping of every fork of
3780 every file in the filesystem, and the filesystem cannot stop.
3781 Therefore, rmap repair foregoes atomicity between scrub and repair.
3782 It combines a :ref:`coordinated inode scanner <iscan>`, :ref:`live update hooks
3783 <liveupdate>`, and an :ref:`in-memory rmap btree <xfbtree>` to complete the
3784 scan for reverse mapping records.
3786 1. Set up an xfbtree to stage rmap records.
3788 2. While holding the locks on the AGI and AGF buffers acquired during the
3789 scrub, generate reverse mappings for all AG metadata: inodes, btrees, CoW
3790 staging extents, and the internal log.
3792 3. Set up an inode scanner.
3794 4. Hook into rmap updates for the AG being repaired so that the live scan data
3795 can receive updates to the rmap btree from the rest of the filesystem during
3798 5. For each space mapping found in either fork of each file scanned,
3799 decide if the mapping matches the AG of interest.
3802 a. Create a btree cursor for the in-memory btree.
3804 b. Use the rmap code to add the record to the in-memory btree.
3806 c. Use the :ref:`special commit function <xfbtree_commit>` to write the
3807 xfbtree changes to the xfile.
3809 6. For each live update received via the hook, decide if the owner has already
3811 If so, apply the live update into the scan data:
3813 a. Create a btree cursor for the in-memory btree.
3815 b. Replay the operation into the in-memory btree.
3817 c. Use the :ref:`special commit function <xfbtree_commit>` to write the
3818 xfbtree changes to the xfile.
3819 This is performed with an empty transaction to avoid changing the
3822 7. When the inode scan finishes, create a new scrub transaction and relock the
3825 8. Compute the new btree geometry using the number of rmap records in the
3826 shadow btree, like all other btree rebuilding functions.
3828 9. Allocate the number of blocks computed in the previous step.
3830 10. Perform the usual btree bulk loading and commit to install the new rmap
3833 11. Reap the old rmap btree blocks as discussed in the case study about how
3834 to :ref:`reap after rmap btree repair <rmap_reap>`.
3836 12. Free the xfbtree now that it not needed.
3838 The proposed patchset is the
3840 <https://git.kernel.org/pub/scm/linux/kernel/git/djwong/xfs-linux.git/log/?h=repair-rmap-btree>`_
3843 Staging Repairs with Temporary Files on Disk
3844 --------------------------------------------
3846 XFS stores a substantial amount of metadata in file forks: directories,
3847 extended attributes, symbolic link targets, free space bitmaps and summary
3848 information for the realtime volume, and quota records.
3849 File forks map 64-bit logical file fork space extents to physical storage space
3850 extents, similar to how a memory management unit maps 64-bit virtual addresses
3851 to physical memory addresses.
3852 Therefore, file-based tree structures (such as directories and extended
3853 attributes) use blocks mapped in the file fork offset address space that point
3854 to other blocks mapped within that same address space, and file-based linear
3855 structures (such as bitmaps and quota records) compute array element offsets in
3856 the file fork offset address space.
3858 Because file forks can consume as much space as the entire filesystem, repairs
3859 cannot be staged in memory, even when a paging scheme is available.
3860 Therefore, online repair of file-based metadata createas a temporary file in
3861 the XFS filesystem, writes a new structure at the correct offsets into the
3862 temporary file, and atomically swaps the fork mappings (and hence the fork
3863 contents) to commit the repair.
3864 Once the repair is complete, the old fork can be reaped as necessary; if the
3865 system goes down during the reap, the iunlink code will delete the blocks
3866 during log recovery.
3868 **Note**: All space usage and inode indices in the filesystem *must* be
3869 consistent to use a temporary file safely!
3870 This dependency is the reason why online repair can only use pageable kernel
3871 memory to stage ondisk space usage information.
3873 Swapping metadata extents with a temporary file requires the owner field of the
3874 block headers to match the file being repaired and not the temporary file. The
3875 directory, extended attribute, and symbolic link functions were all modified to
3876 allow callers to specify owner numbers explicitly.
3878 There is a downside to the reaping process -- if the system crashes during the
3879 reap phase and the fork extents are crosslinked, the iunlink processing will
3880 fail because freeing space will find the extra reverse mappings and abort.
3882 Temporary files created for repair are similar to ``O_TMPFILE`` files created
3884 They are not linked into a directory and the entire file will be reaped when
3885 the last reference to the file is lost.
3886 The key differences are that these files must have no access permission outside
3887 the kernel at all, they must be specially marked to prevent them from being
3888 opened by handle, and they must never be linked into the directory tree.
3890 +--------------------------------------------------------------------------+
3891 | **Historical Sidebar**: |
3892 +--------------------------------------------------------------------------+
3893 | In the initial iteration of file metadata repair, the damaged metadata |
3894 | blocks would be scanned for salvageable data; the extents in the file |
3895 | fork would be reaped; and then a new structure would be built in its |
3897 | This strategy did not survive the introduction of the atomic repair |
3898 | requirement expressed earlier in this document. |
3900 | The second iteration explored building a second structure at a high |
3901 | offset in the fork from the salvage data, reaping the old extents, and |
3902 | using a ``COLLAPSE_RANGE`` operation to slide the new extents into |
3905 | This had many drawbacks: |
3907 | - Array structures are linearly addressed, and the regular filesystem |
3908 | codebase does not have the concept of a linear offset that could be |
3909 | applied to the record offset computation to build an alternate copy. |
3911 | - Extended attributes are allowed to use the entire attr fork offset |
3914 | - Even if repair could build an alternate copy of a data structure in a |
3915 | different part of the fork address space, the atomic repair commit |
3916 | requirement means that online repair would have to be able to perform |
3917 | a log assisted ``COLLAPSE_RANGE`` operation to ensure that the old |
3918 | structure was completely replaced. |
3920 | - A crash after construction of the secondary tree but before the range |
3921 | collapse would leave unreachable blocks in the file fork. |
3922 | This would likely confuse things further. |
3924 | - Reaping blocks after a repair is not a simple operation, and |
3925 | initiating a reap operation from a restarted range collapse operation |
3926 | during log recovery is daunting. |
3928 | - Directory entry blocks and quota records record the file fork offset |
3929 | in the header area of each block. |
3930 | An atomic range collapse operation would have to rewrite this part of |
3931 | each block header. |
3932 | Rewriting a single field in block headers is not a huge problem, but |
3933 | it's something to be aware of. |
3935 | - Each block in a directory or extended attributes btree index contains |
3936 | sibling and child block pointers. |
3937 | Were the atomic commit to use a range collapse operation, each block |
3938 | would have to be rewritten very carefully to preserve the graph |
3940 | Doing this as part of a range collapse means rewriting a large number |
3941 | of blocks repeatedly, which is not conducive to quick repairs. |
3943 | This lead to the introduction of temporary file staging. |
3944 +--------------------------------------------------------------------------+
3946 Using a Temporary File
3947 ``````````````````````
3949 Online repair code should use the ``xrep_tempfile_create`` function to create a
3950 temporary file inside the filesystem.
3951 This allocates an inode, marks the in-core inode private, and attaches it to
3953 These files are hidden from userspace, may not be added to the directory tree,
3954 and must be kept private.
3956 Temporary files only use two inode locks: the IOLOCK and the ILOCK.
3957 The MMAPLOCK is not needed here, because there must not be page faults from
3958 userspace for data fork blocks.
3959 The usage patterns of these two locks are the same as for any other XFS file --
3960 access to file data are controlled via the IOLOCK, and access to file metadata
3961 are controlled via the ILOCK.
3962 Locking helpers are provided so that the temporary file and its lock state can
3963 be cleaned up by the scrub context.
3964 To comply with the nested locking strategy laid out in the :ref:`inode
3965 locking<ilocking>` section, it is recommended that scrub functions use the
3966 xrep_tempfile_ilock*_nowait lock helpers.
3968 Data can be written to a temporary file by two means:
3970 1. ``xrep_tempfile_copyin`` can be used to set the contents of a regular
3971 temporary file from an xfile.
3973 2. The regular directory, symbolic link, and extended attribute functions can
3974 be used to write to the temporary file.
3976 Once a good copy of a data file has been constructed in a temporary file, it
3977 must be conveyed to the file being repaired, which is the topic of the next
3980 The proposed patches are in the
3981 `repair temporary files
3982 <https://git.kernel.org/pub/scm/linux/kernel/git/djwong/xfs-linux.git/log/?h=repair-tempfiles>`_
3985 Atomic Extent Swapping
3986 ----------------------
3988 Once repair builds a temporary file with a new data structure written into
3989 it, it must commit the new changes into the existing file.
3990 It is not possible to swap the inumbers of two files, so instead the new
3991 metadata must replace the old.
3992 This suggests the need for the ability to swap extents, but the existing extent
3993 swapping code used by the file defragmenting tool ``xfs_fsr`` is not sufficient
3994 for online repair because:
3996 a. When the reverse-mapping btree is enabled, the swap code must keep the
3997 reverse mapping information up to date with every exchange of mappings.
3998 Therefore, it can only exchange one mapping per transaction, and each
3999 transaction is independent.
4001 b. Reverse-mapping is critical for the operation of online fsck, so the old
4002 defragmentation code (which swapped entire extent forks in a single
4003 operation) is not useful here.
4005 c. Defragmentation is assumed to occur between two files with identical
4007 For this use case, an incomplete exchange will not result in a user-visible
4008 change in file contents, even if the operation is interrupted.
4010 d. Online repair needs to swap the contents of two files that are by definition
4012 For directory and xattr repairs, the user-visible contents might be the
4013 same, but the contents of individual blocks may be very different.
4015 e. Old blocks in the file may be cross-linked with another structure and must
4016 not reappear if the system goes down mid-repair.
4018 These problems are overcome by creating a new deferred operation and a new type
4019 of log intent item to track the progress of an operation to exchange two file
4021 The new deferred operation type chains together the same transactions used by
4022 the reverse-mapping extent swap code.
4023 The new log item records the progress of the exchange to ensure that once an
4024 exchange begins, it will always run to completion, even there are
4026 The new ``XFS_SB_FEAT_INCOMPAT_LOG_ATOMIC_SWAP`` log-incompatible feature flag
4027 in the superblock protects these new log item records from being replayed on
4030 The proposed patchset is the
4032 <https://git.kernel.org/pub/scm/linux/kernel/git/djwong/xfs-linux.git/log/?h=atomic-file-updates>`_
4035 +--------------------------------------------------------------------------+
4036 | **Sidebar: Using Log-Incompatible Feature Flags** |
4037 +--------------------------------------------------------------------------+
4038 | Starting with XFS v5, the superblock contains a |
4039 | ``sb_features_log_incompat`` field to indicate that the log contains |
4040 | records that might not readable by all kernels that could mount this |
4042 | In short, log incompat features protect the log contents against kernels |
4043 | that will not understand the contents. |
4044 | Unlike the other superblock feature bits, log incompat bits are |
4045 | ephemeral because an empty (clean) log does not need protection. |
4046 | The log cleans itself after its contents have been committed into the |
4047 | filesystem, either as part of an unmount or because the system is |
4049 | Because upper level code can be working on a transaction at the same |
4050 | time that the log cleans itself, it is necessary for upper level code to |
4051 | communicate to the log when it is going to use a log incompatible |
4054 | The log coordinates access to incompatible features through the use of |
4055 | one ``struct rw_semaphore`` for each feature. |
4056 | The log cleaning code tries to take this rwsem in exclusive mode to |
4057 | clear the bit; if the lock attempt fails, the feature bit remains set. |
4058 | Filesystem code signals its intention to use a log incompat feature in a |
4059 | transaction by calling ``xlog_use_incompat_feat``, which takes the rwsem |
4061 | The code supporting a log incompat feature should create wrapper |
4062 | functions to obtain the log feature and call |
4063 | ``xfs_add_incompat_log_feature`` to set the feature bits in the primary |
4065 | The superblock update is performed transactionally, so the wrapper to |
4066 | obtain log assistance must be called just prior to the creation of the |
4067 | transaction that uses the functionality. |
4068 | For a file operation, this step must happen after taking the IOLOCK |
4069 | and the MMAPLOCK, but before allocating the transaction. |
4070 | When the transaction is complete, the ``xlog_drop_incompat_feat`` |
4071 | function is called to release the feature. |
4072 | The feature bit will not be cleared from the superblock until the log |
4075 | Log-assisted extended attribute updates and atomic extent swaps both use |
4076 | log incompat features and provide convenience wrappers around the |
4078 +--------------------------------------------------------------------------+
4080 Mechanics of an Atomic Extent Swap
4081 ``````````````````````````````````
4083 Swapping entire file forks is a complex task.
4084 The goal is to exchange all file fork mappings between two file fork offset
4086 There are likely to be many extent mappings in each fork, and the edges of
4087 the mappings aren't necessarily aligned.
4088 Furthermore, there may be other updates that need to happen after the swap,
4089 such as exchanging file sizes, inode flags, or conversion of fork data to local
4091 This is roughly the format of the new deferred extent swap work item:
4095 struct xfs_swapext_intent {
4096 /* Inodes participating in the operation. */
4097 struct xfs_inode *sxi_ip1;
4098 struct xfs_inode *sxi_ip2;
4100 /* File offset range information. */
4101 xfs_fileoff_t sxi_startoff1;
4102 xfs_fileoff_t sxi_startoff2;
4103 xfs_filblks_t sxi_blockcount;
4105 /* Set these file sizes after the operation, unless negative. */
4106 xfs_fsize_t sxi_isize1;
4107 xfs_fsize_t sxi_isize2;
4109 /* XFS_SWAP_EXT_* log operation flags */
4113 The new log intent item contains enough information to track two logical fork
4114 offset ranges: ``(inode1, startoff1, blockcount)`` and ``(inode2, startoff2,
4116 Each step of a swap operation exchanges the largest file range mapping possible
4117 from one file to the other.
4118 After each step in the swap operation, the two startoff fields are incremented
4119 and the blockcount field is decremented to reflect the progress made.
4120 The flags field captures behavioral parameters such as swapping the attr fork
4121 instead of the data fork and other work to be done after the extent swap.
4122 The two isize fields are used to swap the file size at the end of the operation
4123 if the file data fork is the target of the swap operation.
4125 When the extent swap is initiated, the sequence of operations is as follows:
4127 1. Create a deferred work item for the extent swap.
4128 At the start, it should contain the entirety of the file ranges to be
4131 2. Call ``xfs_defer_finish`` to process the exchange.
4132 This is encapsulated in ``xrep_tempswap_contents`` for scrub operations.
4133 This will log an extent swap intent item to the transaction for the deferred
4134 extent swap work item.
4136 3. Until ``sxi_blockcount`` of the deferred extent swap work item is zero,
4138 a. Read the block maps of both file ranges starting at ``sxi_startoff1`` and
4139 ``sxi_startoff2``, respectively, and compute the longest extent that can
4140 be swapped in a single step.
4141 This is the minimum of the two ``br_blockcount`` s in the mappings.
4142 Keep advancing through the file forks until at least one of the mappings
4143 contains written blocks.
4144 Mutual holes, unwritten extents, and extent mappings to the same physical
4145 space are not exchanged.
4147 For the next few steps, this document will refer to the mapping that came
4148 from file 1 as "map1", and the mapping that came from file 2 as "map2".
4150 b. Create a deferred block mapping update to unmap map1 from file 1.
4152 c. Create a deferred block mapping update to unmap map2 from file 2.
4154 d. Create a deferred block mapping update to map map1 into file 2.
4156 e. Create a deferred block mapping update to map map2 into file 1.
4158 f. Log the block, quota, and extent count updates for both files.
4160 g. Extend the ondisk size of either file if necessary.
4162 h. Log an extent swap done log item for the extent swap intent log item
4163 that was read at the start of step 3.
4165 i. Compute the amount of file range that has just been covered.
4166 This quantity is ``(map1.br_startoff + map1.br_blockcount -
4167 sxi_startoff1)``, because step 3a could have skipped holes.
4169 j. Increase the starting offsets of ``sxi_startoff1`` and ``sxi_startoff2``
4170 by the number of blocks computed in the previous step, and decrease
4171 ``sxi_blockcount`` by the same quantity.
4172 This advances the cursor.
4174 k. Log a new extent swap intent log item reflecting the advanced state of
4177 l. Return the proper error code (EAGAIN) to the deferred operation manager
4178 to inform it that there is more work to be done.
4179 The operation manager completes the deferred work in steps 3b-3e before
4180 moving back to the start of step 3.
4182 4. Perform any post-processing.
4183 This will be discussed in more detail in subsequent sections.
4185 If the filesystem goes down in the middle of an operation, log recovery will
4186 find the most recent unfinished extent swap log intent item and restart from
4188 This is how extent swapping guarantees that an outside observer will either see
4189 the old broken structure or the new one, and never a mismash of both.
4191 Preparation for Extent Swapping
4192 ```````````````````````````````
4194 There are a few things that need to be taken care of before initiating an
4195 atomic extent swap operation.
4196 First, regular files require the page cache to be flushed to disk before the
4197 operation begins, and directio writes to be quiesced.
4198 Like any filesystem operation, extent swapping must determine the maximum
4199 amount of disk space and quota that can be consumed on behalf of both files in
4200 the operation, and reserve that quantity of resources to avoid an unrecoverable
4201 out of space failure once it starts dirtying metadata.
4202 The preparation step scans the ranges of both files to estimate:
4204 - Data device blocks needed to handle the repeated updates to the fork
4206 - Change in data and realtime block counts for both files.
4207 - Increase in quota usage for both files, if the two files do not share the
4208 same set of quota ids.
4209 - The number of extent mappings that will be added to each file.
4210 - Whether or not there are partially written realtime extents.
4211 User programs must never be able to access a realtime file extent that maps
4212 to different extents on the realtime volume, which could happen if the
4213 operation fails to run to completion.
4215 The need for precise estimation increases the run time of the swap operation,
4216 but it is very important to maintain correct accounting.
4217 The filesystem must not run completely out of free space, nor can the extent
4218 swap ever add more extent mappings to a fork than it can support.
4219 Regular users are required to abide the quota limits, though metadata repairs
4220 may exceed quota to resolve inconsistent metadata elsewhere.
4222 Special Features for Swapping Metadata File Extents
4223 ```````````````````````````````````````````````````
4225 Extended attributes, symbolic links, and directories can set the fork format to
4226 "local" and treat the fork as a literal area for data storage.
4227 Metadata repairs must take extra steps to support these cases:
4229 - If both forks are in local format and the fork areas are large enough, the
4230 swap is performed by copying the incore fork contents, logging both forks,
4232 The atomic extent swap mechanism is not necessary, since this can be done
4233 with a single transaction.
4235 - If both forks map blocks, then the regular atomic extent swap is used.
4237 - Otherwise, only one fork is in local format.
4238 The contents of the local format fork are converted to a block to perform the
4240 The conversion to block format must be done in the same transaction that
4241 logs the initial extent swap intent log item.
4242 The regular atomic extent swap is used to exchange the mappings.
4243 Special flags are set on the swap operation so that the transaction can be
4244 rolled one more time to convert the second file's fork back to local format
4245 so that the second file will be ready to go as soon as the ILOCK is dropped.
4247 Extended attributes and directories stamp the owning inode into every block,
4248 but the buffer verifiers do not actually check the inode number!
4249 Although there is no verification, it is still important to maintain
4250 referential integrity, so prior to performing the extent swap, online repair
4251 builds every block in the new data structure with the owner field of the file
4254 After a successful swap operation, the repair operation must reap the old fork
4255 blocks by processing each fork mapping through the standard :ref:`file extent
4256 reaping <reaping>` mechanism that is done post-repair.
4257 If the filesystem should go down during the reap part of the repair, the
4258 iunlink processing at the end of recovery will free both the temporary file and
4259 whatever blocks were not reaped.
4260 However, this iunlink processing omits the cross-link detection of online
4261 repair, and is not completely foolproof.
4263 Swapping Temporary File Extents
4264 ```````````````````````````````
4266 To repair a metadata file, online repair proceeds as follows:
4268 1. Create a temporary repair file.
4270 2. Use the staging data to write out new contents into the temporary repair
4272 The same fork must be written to as is being repaired.
4274 3. Commit the scrub transaction, since the swap estimation step must be
4275 completed before transaction reservations are made.
4277 4. Call ``xrep_tempswap_trans_alloc`` to allocate a new scrub transaction with
4278 the appropriate resource reservations, locks, and fill out a ``struct
4279 xfs_swapext_req`` with the details of the swap operation.
4281 5. Call ``xrep_tempswap_contents`` to swap the contents.
4283 6. Commit the transaction to complete the repair.
4287 Case Study: Repairing the Realtime Summary File
4288 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
4290 In the "realtime" section of an XFS filesystem, free space is tracked via a
4291 bitmap, similar to Unix FFS.
4292 Each bit in the bitmap represents one realtime extent, which is a multiple of
4293 the filesystem block size between 4KiB and 1GiB in size.
4294 The realtime summary file indexes the number of free extents of a given size to
4295 the offset of the block within the realtime free space bitmap where those free
4297 In other words, the summary file helps the allocator find free extents by
4298 length, similar to what the free space by count (cntbt) btree does for the data
4301 The summary file itself is a flat file (with no block headers or checksums!)
4302 partitioned into ``log2(total rt extents)`` sections containing enough 32-bit
4303 counters to match the number of blocks in the rt bitmap.
4304 Each counter records the number of free extents that start in that bitmap block
4305 and can satisfy a power-of-two allocation request.
4307 To check the summary file against the bitmap:
4309 1. Take the ILOCK of both the realtime bitmap and summary files.
4311 2. For each free space extent recorded in the bitmap:
4313 a. Compute the position in the summary file that contains a counter that
4314 represents this free extent.
4316 b. Read the counter from the xfile.
4318 c. Increment it, and write it back to the xfile.
4320 3. Compare the contents of the xfile against the ondisk file.
4322 To repair the summary file, write the xfile contents into the temporary file
4323 and use atomic extent swap to commit the new contents.
4324 The temporary file is then reaped.
4326 The proposed patchset is the
4327 `realtime summary repair
4328 <https://git.kernel.org/pub/scm/linux/kernel/git/djwong/xfs-linux.git/log/?h=repair-rtsummary>`_
4331 Case Study: Salvaging Extended Attributes
4332 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
4334 In XFS, extended attributes are implemented as a namespaced name-value store.
4335 Values are limited in size to 64KiB, but there is no limit in the number of
4337 The attribute fork is unpartitioned, which means that the root of the attribute
4338 structure is always in logical block zero, but attribute leaf blocks, dabtree
4339 index blocks, and remote value blocks are intermixed.
4340 Attribute leaf blocks contain variable-sized records that associate
4341 user-provided names with the user-provided values.
4342 Values larger than a block are allocated separate extents and written there.
4343 If the leaf information expands beyond a single block, a directory/attribute
4344 btree (``dabtree``) is created to map hashes of attribute names to entries
4347 Salvaging extended attributes is done as follows:
4349 1. Walk the attr fork mappings of the file being repaired to find the attribute
4353 a. Walk the attr leaf block to find candidate keys.
4356 1. Check the name for problems, and ignore the name if there are.
4358 2. Retrieve the value.
4359 If that succeeds, add the name and value to the staging xfarray and
4362 2. If the memory usage of the xfarray and xfblob exceed a certain amount of
4363 memory or there are no more attr fork blocks to examine, unlock the file and
4364 add the staged extended attributes to the temporary file.
4366 3. Use atomic extent swapping to exchange the new and old extended attribute
4368 The old attribute blocks are now attached to the temporary file.
4370 4. Reap the temporary file.
4372 The proposed patchset is the
4373 `extended attribute repair
4374 <https://git.kernel.org/pub/scm/linux/kernel/git/djwong/xfs-linux.git/log/?h=repair-xattrs>`_
4380 Fixing directories is difficult with currently available filesystem features,
4381 since directory entries are not redundant.
4382 The offline repair tool scans all inodes to find files with nonzero link count,
4383 and then it scans all directories to establish parentage of those linked files.
4384 Damaged files and directories are zapped, and files with no parent are
4385 moved to the ``/lost+found`` directory.
4386 It does not try to salvage anything.
4388 The best that online repair can do at this time is to read directory data
4389 blocks and salvage any dirents that look plausible, correct link counts, and
4390 move orphans back into the directory tree.
4391 The salvage process is discussed in the case study at the end of this section.
4392 The :ref:`file link count fsck <nlinks>` code takes care of fixing link counts
4393 and moving orphans to the ``/lost+found`` directory.
4395 Case Study: Salvaging Directories
4396 `````````````````````````````````
4398 Unlike extended attributes, directory blocks are all the same size, so
4399 salvaging directories is straightforward:
4401 1. Find the parent of the directory.
4402 If the dotdot entry is not unreadable, try to confirm that the alleged
4403 parent has a child entry pointing back to the directory being repaired.
4404 Otherwise, walk the filesystem to find it.
4406 2. Walk the first partition of data fork of the directory to find the directory
4410 a. Walk the directory data block to find candidate entries.
4411 When an entry is found:
4413 i. Check the name for problems, and ignore the name if there are.
4415 ii. Retrieve the inumber and grab the inode.
4416 If that succeeds, add the name, inode number, and file type to the
4417 staging xfarray and xblob.
4419 3. If the memory usage of the xfarray and xfblob exceed a certain amount of
4420 memory or there are no more directory data blocks to examine, unlock the
4421 directory and add the staged dirents into the temporary directory.
4422 Truncate the staging files.
4424 4. Use atomic extent swapping to exchange the new and old directory structures.
4425 The old directory blocks are now attached to the temporary file.
4427 5. Reap the temporary file.
4429 **Future Work Question**: Should repair revalidate the dentry cache when
4430 rebuilding a directory?
4432 *Answer*: Yes, it should.
4434 In theory it is necessary to scan all dentry cache entries for a directory to
4435 ensure that one of the following apply:
4437 1. The cached dentry reflects an ondisk dirent in the new directory.
4439 2. The cached dentry no longer has a corresponding ondisk dirent in the new
4440 directory and the dentry can be purged from the cache.
4442 3. The cached dentry no longer has an ondisk dirent but the dentry cannot be
4444 This is the problem case.
4446 Unfortunately, the current dentry cache design doesn't provide a means to walk
4447 every child dentry of a specific directory, which makes this a hard problem.
4448 There is no known solution.
4450 The proposed patchset is the
4452 <https://git.kernel.org/pub/scm/linux/kernel/git/djwong/xfs-linux.git/log/?h=repair-dirs>`_
4458 A parent pointer is a piece of file metadata that enables a user to locate the
4459 file's parent directory without having to traverse the directory tree from the
4461 Without them, reconstruction of directory trees is hindered in much the same
4462 way that the historic lack of reverse space mapping information once hindered
4463 reconstruction of filesystem space metadata.
4464 The parent pointer feature, however, makes total directory reconstruction
4467 XFS parent pointers include the dirent name and location of the entry within
4468 the parent directory.
4469 In other words, child files use extended attributes to store pointers to
4470 parents in the form ``(parent_inum, parent_gen, dirent_pos) → (dirent_name)``.
4471 The directory checking process can be strengthened to ensure that the target of
4472 each dirent also contains a parent pointer pointing back to the dirent.
4473 Likewise, each parent pointer can be checked by ensuring that the target of
4474 each parent pointer is a directory and that it contains a dirent matching
4476 Both online and offline repair can use this strategy.
4478 **Note**: The ondisk format of parent pointers is not yet finalized.
4480 +--------------------------------------------------------------------------+
4481 | **Historical Sidebar**: |
4482 +--------------------------------------------------------------------------+
4483 | Directory parent pointers were first proposed as an XFS feature more |
4484 | than a decade ago by SGI. |
4485 | Each link from a parent directory to a child file is mirrored with an |
4486 | extended attribute in the child that could be used to identify the |
4487 | parent directory. |
4488 | Unfortunately, this early implementation had major shortcomings and was |
4489 | never merged into Linux XFS: |
4491 | 1. The XFS codebase of the late 2000s did not have the infrastructure to |
4492 | enforce strong referential integrity in the directory tree. |
4493 | It did not guarantee that a change in a forward link would always be |
4494 | followed up with the corresponding change to the reverse links. |
4496 | 2. Referential integrity was not integrated into offline repair. |
4497 | Checking and repairs were performed on mounted filesystems without |
4498 | taking any kernel or inode locks to coordinate access. |
4499 | It is not clear how this actually worked properly. |
4501 | 3. The extended attribute did not record the name of the directory entry |
4502 | in the parent, so the SGI parent pointer implementation cannot be |
4503 | used to reconnect the directory tree. |
4505 | 4. Extended attribute forks only support 65,536 extents, which means |
4506 | that parent pointer attribute creation is likely to fail at some |
4507 | point before the maximum file link count is achieved. |
4509 | The original parent pointer design was too unstable for something like |
4510 | a file system repair to depend on. |
4511 | Allison Henderson, Chandan Babu, and Catherine Hoang are working on a |
4512 | second implementation that solves all shortcomings of the first. |
4513 | During 2022, Allison introduced log intent items to track physical |
4514 | manipulations of the extended attribute structures. |
4515 | This solves the referential integrity problem by making it possible to |
4516 | commit a dirent update and a parent pointer update in the same |
4518 | Chandan increased the maximum extent counts of both data and attribute |
4519 | forks, thereby ensuring that the extended attribute structure can grow |
4520 | to handle the maximum hardlink count of any file. |
4521 +--------------------------------------------------------------------------+
4523 Case Study: Repairing Directories with Parent Pointers
4524 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
4526 Directory rebuilding uses a :ref:`coordinated inode scan <iscan>` and
4527 a :ref:`directory entry live update hook <liveupdate>` as follows:
4529 1. Set up a temporary directory for generating the new directory structure,
4530 an xfblob for storing entry names, and an xfarray for stashing directory
4533 2. Set up an inode scanner and hook into the directory entry code to receive
4534 updates on directory operations.
4536 3. For each parent pointer found in each file scanned, decide if the parent
4537 pointer references the directory of interest.
4540 a. Stash an addname entry for this dirent in the xfarray for later.
4542 b. When finished scanning that file, flush the stashed updates to the
4543 temporary directory.
4545 4. For each live directory update received via the hook, decide if the child
4546 has already been scanned.
4549 a. Stash an addname or removename entry for this dirent update in the
4551 We cannot write directly to the temporary directory because hook
4552 functions are not allowed to modify filesystem metadata.
4553 Instead, we stash updates in the xfarray and rely on the scanner thread
4554 to apply the stashed updates to the temporary directory.
4556 5. When the scan is complete, atomically swap the contents of the temporary
4557 directory and the directory being repaired.
4558 The temporary directory now contains the damaged directory structure.
4560 6. Reap the temporary directory.
4562 7. Update the dirent position field of parent pointers as necessary.
4563 This may require the queuing of a substantial number of xattr log intent
4566 The proposed patchset is the
4567 `parent pointers directory repair
4568 <https://git.kernel.org/pub/scm/linux/kernel/git/djwong/xfs-linux.git/log/?h=pptrs-online-dir-repair>`_
4571 **Unresolved Question**: How will repair ensure that the ``dirent_pos`` fields
4572 match in the reconstructed directory?
4574 *Answer*: There are a few ways to solve this problem:
4576 1. The field could be designated advisory, since the other three values are
4577 sufficient to find the entry in the parent.
4578 However, this makes indexed key lookup impossible while repairs are ongoing.
4580 2. We could allow creating directory entries at specified offsets, which solves
4581 the referential integrity problem but runs the risk that dirent creation
4582 will fail due to conflicts with the free space in the directory.
4584 These conflicts could be resolved by appending the directory entry and
4585 amending the xattr code to support updating an xattr key and reindexing the
4586 dabtree, though this would have to be performed with the parent directory
4589 3. Same as above, but remove the old parent pointer entry and add a new one
4592 4. Change the ondisk xattr format to ``(parent_inum, name) → (parent_gen)``,
4593 which would provide the attr name uniqueness that we require, without
4594 forcing repair code to update the dirent position.
4595 Unfortunately, this requires changes to the xattr code to support attr
4596 names as long as 263 bytes.
4598 5. Change the ondisk xattr format to ``(parent_inum, hash(name)) →
4599 (name, parent_gen)``.
4600 If the hash is sufficiently resistant to collisions (e.g. sha256) then
4601 this should provide the attr name uniqueness that we require.
4602 Names shorter than 247 bytes could be stored directly.
4604 Discussion is ongoing under the `parent pointers patch deluge
4605 <https://www.spinics.net/lists/linux-xfs/msg69397.html>`_.
4607 Case Study: Repairing Parent Pointers
4608 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
4610 Online reconstruction of a file's parent pointer information works similarly to
4611 directory reconstruction:
4613 1. Set up a temporary file for generating a new extended attribute structure,
4614 an `xfblob<xfblob>` for storing parent pointer names, and an xfarray for
4615 stashing parent pointer updates.
4617 2. Set up an inode scanner and hook into the directory entry code to receive
4618 updates on directory operations.
4620 3. For each directory entry found in each directory scanned, decide if the
4621 dirent references the file of interest.
4624 a. Stash an addpptr entry for this parent pointer in the xfblob and xfarray
4627 b. When finished scanning the directory, flush the stashed updates to the
4628 temporary directory.
4630 4. For each live directory update received via the hook, decide if the parent
4631 has already been scanned.
4634 a. Stash an addpptr or removepptr entry for this dirent update in the
4636 We cannot write parent pointers directly to the temporary file because
4637 hook functions are not allowed to modify filesystem metadata.
4638 Instead, we stash updates in the xfarray and rely on the scanner thread
4639 to apply the stashed parent pointer updates to the temporary file.
4641 5. Copy all non-parent pointer extended attributes to the temporary file.
4643 6. When the scan is complete, atomically swap the attribute fork of the
4644 temporary file and the file being repaired.
4645 The temporary file now contains the damaged extended attribute structure.
4647 7. Reap the temporary file.
4649 The proposed patchset is the
4650 `parent pointers repair
4651 <https://git.kernel.org/pub/scm/linux/kernel/git/djwong/xfs-linux.git/log/?h=pptrs-online-parent-repair>`_
4654 Digression: Offline Checking of Parent Pointers
4655 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
4657 Examining parent pointers in offline repair works differently because corrupt
4658 files are erased long before directory tree connectivity checks are performed.
4659 Parent pointer checks are therefore a second pass to be added to the existing
4660 connectivity checks:
4662 1. After the set of surviving files has been established (i.e. phase 6),
4663 walk the surviving directories of each AG in the filesystem.
4664 This is already performed as part of the connectivity checks.
4666 2. For each directory entry found, record the name in an xfblob, and store
4667 ``(child_ag_inum, parent_inum, parent_gen, dirent_pos)`` tuples in a
4668 per-AG in-memory slab.
4670 3. For each AG in the filesystem,
4672 a. Sort the per-AG tuples in order of child_ag_inum, parent_inum, and
4675 b. For each inode in the AG,
4677 1. Scan the inode for parent pointers.
4678 Record the names in a per-file xfblob, and store ``(parent_inum,
4679 parent_gen, dirent_pos)`` tuples in a per-file slab.
4681 2. Sort the per-file tuples in order of parent_inum, and dirent_pos.
4683 3. Position one slab cursor at the start of the inode's records in the
4685 This should be trivial since the per-AG tuples are in child inumber
4688 4. Position a second slab cursor at the start of the per-file tuple slab.
4690 5. Iterate the two cursors in lockstep, comparing the parent_ino and
4691 dirent_pos fields of the records under each cursor.
4693 a. Tuples in the per-AG list but not the per-file list are missing and
4694 need to be written to the inode.
4696 b. Tuples in the per-file list but not the per-AG list are dangling
4697 and need to be removed from the inode.
4699 c. For tuples in both lists, update the parent_gen and name components
4700 of the parent pointer if necessary.
4702 4. Move on to examining link counts, as we do today.
4704 The proposed patchset is the
4705 `offline parent pointers repair
4706 <https://git.kernel.org/pub/scm/linux/kernel/git/djwong/xfsprogs-dev.git/log/?h=pptrs-repair>`_
4709 Rebuilding directories from parent pointers in offline repair is very
4710 challenging because it currently uses a single-pass scan of the filesystem
4711 during phase 3 to decide which files are corrupt enough to be zapped.
4712 This scan would have to be converted into a multi-pass scan:
4714 1. The first pass of the scan zaps corrupt inodes, forks, and attributes
4715 much as it does now.
4716 Corrupt directories are noted but not zapped.
4718 2. The next pass records parent pointers pointing to the directories noted
4719 as being corrupt in the first pass.
4720 This second pass may have to happen after the phase 4 scan for duplicate
4721 blocks, if phase 4 is also capable of zapping directories.
4723 3. The third pass resets corrupt directories to an empty shortform directory.
4724 Free space metadata has not been ensured yet, so repair cannot yet use the
4725 directory building code in libxfs.
4727 4. At the start of phase 6, space metadata have been rebuilt.
4728 Use the parent pointer information recorded during step 2 to reconstruct
4729 the dirents and add them to the now-empty directories.
4731 This code has not yet been constructed.
4738 Filesystems present files as a directed, and hopefully acyclic, graph.
4739 In other words, a tree.
4740 The root of the filesystem is a directory, and each entry in a directory points
4741 downwards either to more subdirectories or to non-directory files.
4742 Unfortunately, a disruption in the directory graph pointers result in a
4743 disconnected graph, which makes files impossible to access via regular path
4746 Without parent pointers, the directory parent pointer online scrub code can
4747 detect a dotdot entry pointing to a parent directory that doesn't have a link
4748 back to the child directory and the file link count checker can detect a file
4749 that isn't pointed to by any directory in the filesystem.
4750 If such a file has a positive link count, the file is an orphan.
4752 With parent pointers, directories can be rebuilt by scanning parent pointers
4753 and parent pointers can be rebuilt by scanning directories.
4754 This should reduce the incidence of files ending up in ``/lost+found``.
4756 When orphans are found, they should be reconnected to the directory tree.
4757 Offline fsck solves the problem by creating a directory ``/lost+found`` to
4758 serve as an orphanage, and linking orphan files into the orphanage by using the
4759 inumber as the name.
4760 Reparenting a file to the orphanage does not reset any of its permissions or
4763 This process is more involved in the kernel than it is in userspace.
4764 The directory and file link count repair setup functions must use the regular
4765 VFS mechanisms to create the orphanage directory with all the necessary
4766 security attributes and dentry cache entries, just like a regular directory
4769 Orphaned files are adopted by the orphanage as follows:
4771 1. Call ``xrep_orphanage_try_create`` at the start of the scrub setup function
4772 to try to ensure that the lost and found directory actually exists.
4773 This also attaches the orphanage directory to the scrub context.
4775 2. If the decision is made to reconnect a file, take the IOLOCK of both the
4776 orphanage and the file being reattached.
4777 The ``xrep_orphanage_iolock_two`` function follows the inode locking
4778 strategy discussed earlier.
4780 3. Call ``xrep_orphanage_compute_blkres`` and ``xrep_orphanage_compute_name``
4781 to compute the new name in the orphanage and the block reservation required.
4783 4. Use ``xrep_orphanage_adoption_prep`` to reserve resources to the repair
4786 5. Call ``xrep_orphanage_adopt`` to reparent the orphaned file into the lost
4787 and found, and update the kernel dentry cache.
4789 The proposed patches are in the
4791 <https://git.kernel.org/pub/scm/linux/kernel/git/djwong/xfs-linux.git/log/?h=repair-orphanage>`_
4794 6. Userspace Algorithms and Data Structures
4795 ===========================================
4797 This section discusses the key algorithms and data structures of the userspace
4798 program, ``xfs_scrub``, that provide the ability to drive metadata checks and
4799 repairs in the kernel, verify file data, and look for other potential problems.
4806 Recall the :ref:`phases of fsck work<scrubphases>` outlined earlier.
4807 That structure follows naturally from the data dependencies designed into the
4808 filesystem from its beginnings in 1993.
4809 In XFS, there are several groups of metadata dependencies:
4811 a. Filesystem summary counts depend on consistency within the inode indices,
4812 the allocation group space btrees, and the realtime volume space
4815 b. Quota resource counts depend on consistency within the quota file data
4816 forks, inode indices, inode records, and the forks of every file on the
4819 c. The naming hierarchy depends on consistency within the directory and
4820 extended attribute structures.
4821 This includes file link counts.
4823 d. Directories, extended attributes, and file data depend on consistency within
4824 the file forks that map directory and extended attribute data to physical
4827 e. The file forks depends on consistency within inode records and the space
4828 metadata indices of the allocation groups and the realtime volume.
4829 This includes quota and realtime metadata files.
4831 f. Inode records depends on consistency within the inode metadata indices.
4833 g. Realtime space metadata depend on the inode records and data forks of the
4834 realtime metadata inodes.
4836 h. The allocation group metadata indices (free space, inodes, reference count,
4837 and reverse mapping btrees) depend on consistency within the AG headers and
4838 between all the AG metadata btrees.
4840 i. ``xfs_scrub`` depends on the filesystem being mounted and kernel support
4841 for online fsck functionality.
4843 Therefore, a metadata dependency graph is a convenient way to schedule checking
4844 operations in the ``xfs_scrub`` program:
4846 - Phase 1 checks that the provided path maps to an XFS filesystem and detect
4847 the kernel's scrubbing abilities, which validates group (i).
4849 - Phase 2 scrubs groups (g) and (h) in parallel using a threaded workqueue.
4851 - Phase 3 scans inodes in parallel.
4852 For each inode, groups (f), (e), and (d) are checked, in that order.
4854 - Phase 4 repairs everything in groups (i) through (d) so that phases 5 and 6
4857 - Phase 5 starts by checking groups (b) and (c) in parallel before moving on
4860 - Phase 6 depends on groups (i) through (b) to find file data blocks to verify,
4861 to read them, and to report which blocks of which files are affected.
4863 - Phase 7 checks group (a), having validated everything else.
4865 Notice that the data dependencies between groups are enforced by the structure
4866 of the program flow.
4868 Parallel Inode Scans
4869 --------------------
4871 An XFS filesystem can easily contain hundreds of millions of inodes.
4872 Given that XFS targets installations with large high-performance storage,
4873 it is desirable to scrub inodes in parallel to minimize runtime, particularly
4874 if the program has been invoked manually from a command line.
4875 This requires careful scheduling to keep the threads as evenly loaded as
4878 Early iterations of the ``xfs_scrub`` inode scanner naïvely created a single
4879 workqueue and scheduled a single workqueue item per AG.
4880 Each workqueue item walked the inode btree (with ``XFS_IOC_INUMBERS``) to find
4881 inode chunks and then called bulkstat (``XFS_IOC_BULKSTAT``) to gather enough
4882 information to construct file handles.
4883 The file handle was then passed to a function to generate scrub items for each
4884 metadata object of each inode.
4885 This simple algorithm leads to thread balancing problems in phase 3 if the
4886 filesystem contains one AG with a few large sparse files and the rest of the
4887 AGs contain many smaller files.
4888 The inode scan dispatch function was not sufficiently granular; it should have
4889 been dispatching at the level of individual inodes, or, to constrain memory
4890 consumption, inode btree records.
4892 Thanks to Dave Chinner, bounded workqueues in userspace enable ``xfs_scrub`` to
4893 avoid this problem with ease by adding a second workqueue.
4894 Just like before, the first workqueue is seeded with one workqueue item per AG,
4895 and it uses INUMBERS to find inode btree chunks.
4896 The second workqueue, however, is configured with an upper bound on the number
4897 of items that can be waiting to be run.
4898 Each inode btree chunk found by the first workqueue's workers are queued to the
4899 second workqueue, and it is this second workqueue that queries BULKSTAT,
4900 creates a file handle, and passes it to a function to generate scrub items for
4901 each metadata object of each inode.
4902 If the second workqueue is too full, the workqueue add function blocks the
4903 first workqueue's workers until the backlog eases.
4904 This doesn't completely solve the balancing problem, but reduces it enough to
4905 move on to more pressing issues.
4907 The proposed patchsets are the scrub
4909 <https://git.kernel.org/pub/scm/linux/kernel/git/djwong/xfsprogs-dev.git/log/?h=scrub-performance-tweaks>`_
4911 `inode scan rebalance
4912 <https://git.kernel.org/pub/scm/linux/kernel/git/djwong/xfsprogs-dev.git/log/?h=scrub-iscan-rebalance>`_
4920 During phase 2, corruptions and inconsistencies reported in any AGI header or
4921 inode btree are repaired immediately, because phase 3 relies on proper
4922 functioning of the inode indices to find inodes to scan.
4923 Failed repairs are rescheduled to phase 4.
4924 Problems reported in any other space metadata are deferred to phase 4.
4925 Optimization opportunities are always deferred to phase 4, no matter their
4928 During phase 3, corruptions and inconsistencies reported in any part of a
4929 file's metadata are repaired immediately if all space metadata were validated
4931 Repairs that fail or cannot be repaired immediately are scheduled for phase 4.
4933 In the original design of ``xfs_scrub``, it was thought that repairs would be
4934 so infrequent that the ``struct xfs_scrub_metadata`` objects used to
4935 communicate with the kernel could also be used as the primary object to
4937 With recent increases in the number of optimizations possible for a given
4938 filesystem object, it became much more memory-efficient to track all eligible
4939 repairs for a given filesystem object with a single repair item.
4940 Each repair item represents a single lockable object -- AGs, metadata files,
4941 individual inodes, or a class of summary information.
4943 Phase 4 is responsible for scheduling a lot of repair work in as quick a
4944 manner as is practical.
4945 The :ref:`data dependencies <scrubcheck>` outlined earlier still apply, which
4946 means that ``xfs_scrub`` must try to complete the repair work scheduled by
4947 phase 2 before trying repair work scheduled by phase 3.
4948 The repair process is as follows:
4950 1. Start a round of repair with a workqueue and enough workers to keep the CPUs
4951 as busy as the user desires.
4953 a. For each repair item queued by phase 2,
4955 i. Ask the kernel to repair everything listed in the repair item for a
4956 given filesystem object.
4958 ii. Make a note if the kernel made any progress in reducing the number
4959 of repairs needed for this object.
4961 iii. If the object no longer requires repairs, revalidate all metadata
4962 associated with this object.
4963 If the revalidation succeeds, drop the repair item.
4964 If not, requeue the item for more repairs.
4966 b. If any repairs were made, jump back to 1a to retry all the phase 2 items.
4968 c. For each repair item queued by phase 3,
4970 i. Ask the kernel to repair everything listed in the repair item for a
4971 given filesystem object.
4973 ii. Make a note if the kernel made any progress in reducing the number
4974 of repairs needed for this object.
4976 iii. If the object no longer requires repairs, revalidate all metadata
4977 associated with this object.
4978 If the revalidation succeeds, drop the repair item.
4979 If not, requeue the item for more repairs.
4981 d. If any repairs were made, jump back to 1c to retry all the phase 3 items.
4983 2. If step 1 made any repair progress of any kind, jump back to step 1 to start
4984 another round of repair.
4986 3. If there are items left to repair, run them all serially one more time.
4987 Complain if the repairs were not successful, since this is the last chance
4990 Corruptions and inconsistencies encountered during phases 5 and 7 are repaired
4992 Corrupt file data blocks reported by phase 6 cannot be recovered by the
4995 The proposed patchsets are the
4996 `repair warning improvements
4997 <https://git.kernel.org/pub/scm/linux/kernel/git/djwong/xfsprogs-dev.git/log/?h=scrub-better-repair-warnings>`_,
4999 `repair data dependency
5000 <https://git.kernel.org/pub/scm/linux/kernel/git/djwong/xfsprogs-dev.git/log/?h=scrub-repair-data-deps>`_
5003 <https://git.kernel.org/pub/scm/linux/kernel/git/djwong/xfsprogs-dev.git/log/?h=scrub-object-tracking>`_,
5006 <https://git.kernel.org/pub/scm/linux/kernel/git/djwong/xfsprogs-dev.git/log/?h=scrub-repair-scheduling>`_
5009 Checking Names for Confusable Unicode Sequences
5010 -----------------------------------------------
5012 If ``xfs_scrub`` succeeds in validating the filesystem metadata by the end of
5013 phase 4, it moves on to phase 5, which checks for suspicious looking names in
5015 These names consist of the filesystem label, names in directory entries, and
5016 the names of extended attributes.
5017 Like most Unix filesystems, XFS imposes the sparest of constraints on the
5020 - Slashes and null bytes are not allowed in directory entries.
5022 - Null bytes are not allowed in userspace-visible extended attributes.
5024 - Null bytes are not allowed in the filesystem label.
5026 Directory entries and attribute keys store the length of the name explicitly
5027 ondisk, which means that nulls are not name terminators.
5028 For this section, the term "naming domain" refers to any place where names are
5029 presented together -- all the names in a directory, or all the attributes of a
5032 Although the Unix naming constraints are very permissive, the reality of most
5033 modern-day Linux systems is that programs work with Unicode character code
5034 points to support international languages.
5035 These programs typically encode those code points in UTF-8 when interfacing
5036 with the C library because the kernel expects null-terminated names.
5037 In the common case, therefore, names found in an XFS filesystem are actually
5038 UTF-8 encoded Unicode data.
5040 To maximize its expressiveness, the Unicode standard defines separate control
5041 points for various characters that render similarly or identically in writing
5042 systems around the world.
5043 For example, the character "Cyrillic Small Letter A" U+0430 "а" often renders
5044 identically to "Latin Small Letter A" U+0061 "a".
5046 The standard also permits characters to be constructed in multiple ways --
5047 either by using a defined code point, or by combining one code point with
5048 various combining marks.
5049 For example, the character "Angstrom Sign U+212B "Å" can also be expressed
5050 as "Latin Capital Letter A" U+0041 "A" followed by "Combining Ring Above"
5052 Both sequences render identically.
5054 Like the standards that preceded it, Unicode also defines various control
5055 characters to alter the presentation of text.
5056 For example, the character "Right-to-Left Override" U+202E can trick some
5057 programs into rendering "moo\\xe2\\x80\\xaegnp.txt" as "mootxt.png".
5058 A second category of rendering problems involves whitespace characters.
5059 If the character "Zero Width Space" U+200B is encountered in a file name, the
5060 name will render identically to a name that does not have the zero width
5063 If two names within a naming domain have different byte sequences but render
5064 identically, a user may be confused by it.
5065 The kernel, in its indifference to upper level encoding schemes, permits this.
5066 Most filesystem drivers persist the byte sequence names that are given to them
5069 Techniques for detecting confusable names are explained in great detail in
5070 sections 4 and 5 of the
5071 `Unicode Security Mechanisms <https://unicode.org/reports/tr39/>`_
5073 When ``xfs_scrub`` detects UTF-8 encoding in use on a system, it uses the
5074 Unicode normalization form NFD in conjunction with the confusable name
5075 detection component of
5076 `libicu <https://github.com/unicode-org/icu>`_
5077 to identify names with a directory or within a file's extended attributes that
5078 could be confused for each other.
5079 Names are also checked for control characters, non-rendering characters, and
5080 mixing of bidirectional characters.
5081 All of these potential issues are reported to the system administrator during
5084 Media Verification of File Data Extents
5085 ---------------------------------------
5087 The system administrator can elect to initiate a media scan of all file data
5089 This scan after validation of all filesystem metadata (except for the summary
5090 counters) as phase 6.
5091 The scan starts by calling ``FS_IOC_GETFSMAP`` to scan the filesystem space map
5092 to find areas that are allocated to file data fork extents.
5093 Gaps between data fork extents that are smaller than 64k are treated as if
5094 they were data fork extents to reduce the command setup overhead.
5095 When the space map scan accumulates a region larger than 32MB, a media
5096 verification request is sent to the disk as a directio read of the raw block
5099 If the verification read fails, ``xfs_scrub`` retries with single-block reads
5100 to narrow down the failure to the specific region of the media and recorded.
5101 When it has finished issuing verification requests, it again uses the space
5102 mapping ioctl to map the recorded media errors back to metadata structures
5103 and report what has been lost.
5104 For media errors in blocks owned by files, parent pointers can be used to
5105 construct file paths from inode numbers for user-friendly reporting.
5107 7. Conclusion and Future Work
5108 =============================
5110 It is hoped that the reader of this document has followed the designs laid out
5111 in this document and now has some familiarity with how XFS performs online
5112 rebuilding of its metadata indices, and how filesystem users can interact with
5114 Although the scope of this work is daunting, it is hoped that this guide will
5115 make it easier for code readers to understand what has been built, for whom it
5116 has been built, and why.
5117 Please feel free to contact the XFS mailing list with questions.
5122 As discussed earlier, a second frontend to the atomic extent swap mechanism is
5123 a new ioctl call that userspace programs can use to commit updates to files
5125 This frontend has been out for review for several years now, though the
5126 necessary refinements to online repair and lack of customer demand mean that
5127 the proposal has not been pushed very hard.
5129 Extent Swapping with Regular User Files
5130 ```````````````````````````````````````
5132 As mentioned earlier, XFS has long had the ability to swap extents between
5133 files, which is used almost exclusively by ``xfs_fsr`` to defragment files.
5134 The earliest form of this was the fork swap mechanism, where the entire
5135 contents of data forks could be exchanged between two files by exchanging the
5136 raw bytes in each inode fork's immediate area.
5137 When XFS v5 came along with self-describing metadata, this old mechanism grew
5138 some log support to continue rewriting the owner fields of BMBT blocks during
5140 When the reverse mapping btree was later added to XFS, the only way to maintain
5141 the consistency of the fork mappings with the reverse mapping index was to
5142 develop an iterative mechanism that used deferred bmap and rmap operations to
5143 swap mappings one at a time.
5144 This mechanism is identical to steps 2-3 from the procedure above except for
5145 the new tracking items, because the atomic extent swap mechanism is an
5146 iteration of an existing mechanism and not something totally novel.
5147 For the narrow case of file defragmentation, the file contents must be
5148 identical, so the recovery guarantees are not much of a gain.
5150 Atomic extent swapping is much more flexible than the existing swapext
5151 implementations because it can guarantee that the caller never sees a mix of
5152 old and new contents even after a crash, and it can operate on two arbitrary
5154 The extra flexibility enables several new use cases:
5156 - **Atomic commit of file writes**: A userspace process opens a file that it
5158 Next, it opens a temporary file and calls the file clone operation to reflink
5159 the first file's contents into the temporary file.
5160 Writes to the original file should instead be written to the temporary file.
5161 Finally, the process calls the atomic extent swap system call
5162 (``FIEXCHANGE_RANGE``) to exchange the file contents, thereby committing all
5163 of the updates to the original file, or none of them.
5165 .. _swapext_if_unchanged:
5167 - **Transactional file updates**: The same mechanism as above, but the caller
5168 only wants the commit to occur if the original file's contents have not
5170 To make this happen, the calling process snapshots the file modification and
5171 change timestamps of the original file before reflinking its data to the
5173 When the program is ready to commit the changes, it passes the timestamps
5174 into the kernel as arguments to the atomic extent swap system call.
5175 The kernel only commits the changes if the provided timestamps match the
5178 - **Emulation of atomic block device writes**: Export a block device with a
5179 logical sector size matching the filesystem block size to force all writes
5180 to be aligned to the filesystem block size.
5181 Stage all writes to a temporary file, and when that is complete, call the
5182 atomic extent swap system call with a flag to indicate that holes in the
5183 temporary file should be ignored.
5184 This emulates an atomic device write in software, and can support arbitrary
5190 As it turns out, the :ref:`refactoring <scrubrepair>` of repair items mentioned
5191 earlier was a catalyst for enabling a vectorized scrub system call.
5192 Since 2018, the cost of making a kernel call has increased considerably on some
5193 systems to mitigate the effects of speculative execution attacks.
5194 This incentivizes program authors to make as few system calls as possible to
5195 reduce the number of times an execution path crosses a security boundary.
5197 With vectorized scrub, userspace pushes to the kernel the identity of a
5198 filesystem object, a list of scrub types to run against that object, and a
5199 simple representation of the data dependencies between the selected scrub
5201 The kernel executes as much of the caller's plan as it can until it hits a
5202 dependency that cannot be satisfied due to a corruption, and tells userspace
5203 how much was accomplished.
5204 It is hoped that ``io_uring`` will pick up enough of this functionality that
5205 online fsck can use that instead of adding a separate vectored scrub system
5208 The relevant patchsets are the
5209 `kernel vectorized scrub
5210 <https://git.kernel.org/pub/scm/linux/kernel/git/djwong/xfs-linux.git/log/?h=vectorized-scrub>`_
5212 `userspace vectorized scrub
5213 <https://git.kernel.org/pub/scm/linux/kernel/git/djwong/xfsprogs-dev.git/log/?h=vectorized-scrub>`_
5216 Quality of Service Targets for Scrub
5217 ------------------------------------
5219 One serious shortcoming of the online fsck code is that the amount of time that
5220 it can spend in the kernel holding resource locks is basically unbounded.
5221 Userspace is allowed to send a fatal signal to the process which will cause
5222 ``xfs_scrub`` to exit when it reaches a good stopping point, but there's no way
5223 for userspace to provide a time budget to the kernel.
5224 Given that the scrub codebase has helpers to detect fatal signals, it shouldn't
5225 be too much work to allow userspace to specify a timeout for a scrub/repair
5226 operation and abort the operation if it exceeds budget.
5227 However, most repair functions have the property that once they begin to touch
5228 ondisk metadata, the operation cannot be cancelled cleanly, after which a QoS
5229 timeout is no longer useful.
5231 Defragmenting Free Space
5232 ------------------------
5234 Over the years, many XFS users have requested the creation of a program to
5235 clear a portion of the physical storage underlying a filesystem so that it
5236 becomes a contiguous chunk of free space.
5237 Call this free space defragmenter ``clearspace`` for short.
5239 The first piece the ``clearspace`` program needs is the ability to read the
5240 reverse mapping index from userspace.
5241 This already exists in the form of the ``FS_IOC_GETFSMAP`` ioctl.
5242 The second piece it needs is a new fallocate mode
5243 (``FALLOC_FL_MAP_FREE_SPACE``) that allocates the free space in a region and
5245 Call this file the "space collector" file.
5246 The third piece is the ability to force an online repair.
5248 To clear all the metadata out of a portion of physical storage, clearspace
5249 uses the new fallocate map-freespace call to map any free space in that region
5250 to the space collector file.
5251 Next, clearspace finds all metadata blocks in that region by way of
5252 ``GETFSMAP`` and issues forced repair requests on the data structure.
5253 This often results in the metadata being rebuilt somewhere that is not being
5255 After each relocation, clearspace calls the "map free space" function again to
5256 collect any newly freed space in the region being cleared.
5258 To clear all the file data out of a portion of the physical storage, clearspace
5259 uses the FSMAP information to find relevant file data blocks.
5260 Having identified a good target, it uses the ``FICLONERANGE`` call on that part
5261 of the file to try to share the physical space with a dummy file.
5262 Cloning the extent means that the original owners cannot overwrite the
5263 contents; any changes will be written somewhere else via copy-on-write.
5264 Clearspace makes its own copy of the frozen extent in an area that is not being
5265 cleared, and uses ``FIEDEUPRANGE`` (or the :ref:`atomic extent swap
5266 <swapext_if_unchanged>` feature) to change the target file's data extent
5267 mapping away from the area being cleared.
5268 When all other mappings have been moved, clearspace reflinks the space into the
5269 space collector file so that it becomes unavailable.
5271 There are further optimizations that could apply to the above algorithm.
5272 To clear a piece of physical storage that has a high sharing factor, it is
5273 strongly desirable to retain this sharing factor.
5274 In fact, these extents should be moved first to maximize sharing factor after
5275 the operation completes.
5276 To make this work smoothly, clearspace needs a new ioctl
5277 (``FS_IOC_GETREFCOUNTS``) to report reference count information to userspace.
5278 With the refcount information exposed, clearspace can quickly find the longest,
5279 most shared data extents in the filesystem, and target them first.
5281 **Future Work Question**: How might the filesystem move inode chunks?
5283 *Answer*: To move inode chunks, Dave Chinner constructed a prototype program
5284 that creates a new file with the old contents and then locklessly runs around
5285 the filesystem updating directory entries.
5286 The operation cannot complete if the filesystem goes down.
5287 That problem isn't totally insurmountable: create an inode remapping table
5288 hidden behind a jump label, and a log item that tracks the kernel walking the
5289 filesystem to update directory entries.
5290 The trouble is, the kernel can't do anything about open files, since it cannot
5293 **Future Work Question**: Can static keys be used to minimize the cost of
5294 supporting ``revoke()`` on XFS files?
5297 Until the first revocation, the bailout code need not be in the call path at
5300 The relevant patchsets are the
5301 `kernel freespace defrag
5302 <https://git.kernel.org/pub/scm/linux/kernel/git/djwong/xfs-linux.git/log/?h=defrag-freespace>`_
5304 `userspace freespace defrag
5305 <https://git.kernel.org/pub/scm/linux/kernel/git/djwong/xfsprogs-dev.git/log/?h=defrag-freespace>`_
5308 Shrinking Filesystems
5309 ---------------------
5311 Removing the end of the filesystem ought to be a simple matter of evacuating
5312 the data and metadata at the end of the filesystem, and handing the freed space
5314 That requires an evacuation of the space at end of the filesystem, which is a
5315 use of free space defragmentation!