In kernel use of migrate_pages()¶

1. Remove folios from the LRU.

   Lists of folios to be migrated are generated by scanning overfolios and moving them into lists. This is done bycallingfolio_isolate_lru().Callingfolio_isolate_lru() increases the references to the folioso that it cannot vanish while the folio migration occurs.It also prevents the swapper or other scans from encounteringthe folio.

2. We need to have a function of type new_folio_t that can bepassed tomigrate_pages(). This function should figure outhow to allocate the correct new folio given the old folio.

3. Themigrate_pages() function is called which attemptsto do the migration. It will call the function to allocatethe new folio for each folio that is considered for moving.

How migrate_pages() works¶

migrate_pages() does several passes over its list of folios. A folio is movedif all references to a folio are removable at the time. The folio hasalready been removed from the LRU viafolio_isolate_lru() and the refcountis increased so that the folio cannot be freed while folio migration occurs.

Steps:

1. Lock the page to be migrated.

2. Ensure that writeback is complete.

3. Lock the new page that we want to move to. It is locked so that accesses tothis (not yet up-to-date) page immediately block while the move is in progress.

4. All the page table references to the page are converted to migrationentries. This decreases the mapcount of a page. If the resultingmapcount is not zero then we do not migrate the page. All user spaceprocesses that attempt to access the page will now wait on the page lockor wait for the migration page table entry to be removed.

5. The i_pages lock is taken. This will cause all processes tryingto access the page via the mapping to block on the spinlock.

6. The refcount of the page is examined and we back out if references remain.Otherwise, we know that we are the only one referencing this page.

7. The radix tree is checked and if it does not contain the pointer to thispage then we back out because someone else modified the radix tree.

8. The new page is prepped with some settings from the old page so thataccesses to the new page will discover a page with the correct settings.

9. The radix tree is changed to point to the new page.

10. The reference count of the old page is dropped because the address spacereference is gone. A reference to the new page is established becausethe new page is referenced by the address space.

11. The i_pages lock is dropped. With that lookups in the mappingbecome possible again. Processes will move from spinning on the lockto sleeping on the locked new page.

12. The page contents are copied to the new page.

13. The remaining page flags are copied to the new page.

14. The old page flags are cleared to indicate that the page doesnot provide any information anymore.

15. Queued up writeback on the new page is triggered.

16. If migration entries were inserted into the page table, then replace themwith real ptes. Doing so will enable access for user space processes notalready waiting for the page lock.

17. The page locks are dropped from the old and new page.Processes waiting on the page lock will redo their page faultsand will reach the new page.

18. The new page is moved to the LRU and can be scanned by the swapper,etc. again.

movable_ops page migration¶

Selected typed, non-folio pages (e.g., pages inflated in a memory balloon,zsmalloc pages) can be migrated using the movable_ops migration framework.

The “structmovable_operations” provide callbacks specific to a page typefor isolating, migrating and un-isolating (putback) these pages.

Once a page is indicated as having movable_ops, that condition must notchange until the page was freed back to the buddy. This includes notchanging/clearing the page type and not changing/clearing thePG_movable_ops page flag.

Arbitrary drivers cannot currently make use of this framework, as itrequires:

1. a page type

2. indicating them as possibly having movable_ops inpage_has_movable_ops()based on the page type

3. returning the movable_ops frompage_movable_ops() based on the pagetype

4. not reusing the PG_movable_ops and PG_movable_ops_isolated page flagsfor other purposes

For example, balloon drivers can make use of this framework through theballoon-compaction infrastructure residing in the core kernel.

Monitoring Migration¶

The following events (counters) can be used to monitor page migration.

1. PGMIGRATE_SUCCESS: Normal page migration success. Each count means that apage was migrated. If the page was a non-THP and non-hugetlb page, thenthis counter is increased by one. If the page was a THP or hugetlb, thenthis counter is increased by the number of THP or hugetlb subpages.For example, migration of a single 2MB THP that has 4KB-size base pages(subpages) will cause this counter to increase by 512.

2. PGMIGRATE_FAIL: Normal page migration failure. Same counting rules as forPGMIGRATE_SUCCESS, above: this will be increased by the number of subpages,if it was a THP or hugetlb.

3. THP_MIGRATION_SUCCESS: A THP was migrated without being split.

4. THP_MIGRATION_FAIL: A THP could not be migrated nor it could be split.

5. THP_MIGRATION_SPLIT: A THP was migrated, but not as such: first, the THP hadto be split. After splitting, a migration retry was used for its sub-pages.

THP_MIGRATION_* events also update the appropriate PGMIGRATE_SUCCESS orPGMIGRATE_FAIL events. For example, a THP migration failure will cause bothTHP_MIGRATION_FAIL and PGMIGRATE_FAIL to increase.

Christoph Lameter, May 8, 2006.Minchan Kim, Mar 28, 2016.

structmovable_operations¶

Driver page migration

struct movable_operations {    bool (*isolate_page)(struct page *, isolate_mode_t);    int (*migrate_page)(struct page *dst, struct page *src, enum migrate_mode);    void (*putback_page)(struct page *);};