Setup x86_64 Paging to Remap Kernel (1/2) | UEFI Dev (in C)
60Setting up x86_64 4 level paging to use for remapping memory and the kernel.
Part 2 will add a new GDT and do more remapping & bug fixes to call the kernel at a new higher memory address. Split it up for 2 ~2hr halves.
I also never said what paging was to begin with in this video; paging is a way to "map" a virtual address (anything in the full address space, like addresses 0-4GB for x86 32 bit cpus without address extensions) to a physical address (your actual RAM amount e.g. your 2GB or 32GB sticks of RAM) via page tables, which are data structures/arrays/maps that the cpu looks through to check if an address is valid, and what permissions that address has, and where that address actually is (the physical address that is "mapped" there).
You can use paging to use "any" (virtual) address mapped to "any" physical address. That's
useful for tasks, processes, separating address spaces, reusing the same address for multiple programs, have multiple programs loaded in different physical places in RAM, have different permissions for different address ranges like execute vs. read/write, and other things.
It effectively expands your physical RAM amount to the full address range available on your processor, you can act like you have 256TB of RAM instead of 8GB of RAM, but you have to "fake it" by moving virtual addresses around and map/unmap those to the actual 8GB RAM addresses, in the page tables.
Errata:
- The UEFI spec says OS loaders are free to use 0x80000000..0xFFFFFFFF, which is 2GB-4GB, not all of upper memory. This is physical memory, not virtual, and you probably should not assume you can use any range of physical memory outside of that range without otherwise checking e.g. via the EFI memory map.
- By using AllocateAddress or AllocateMaxAddress for a loaded program or a kernel buffer, I was thinking you could directly allocate to a high virtual memory address, but no, it'd be for a specific physical address, probably at/under 4GB.
However, you could still rely on allocating to specific addresss or a max upper range instead of handling it more dynamically, which could makes things simpler and work better with normal OS dev linker scripts and things.
Links:
https://uefi.org/specifications (uefi & other specifications)
https://wiki.osdev.org/Paging
https://en.wikipedia.org/wiki/X86-64#Virtual_address_space_details (and the physical address details, and other sections)
https://blog.llandsmeer.com/tech/2019/07/21/uefi-x64-userland.html (great blog, I based my paging code off of this, it helped a lot)
UEFI Programming playlist:
https://www.youtube.com/playlist?list=PLT7NbkyNWaqZYHNLtOZ1MNxOt8myP5K0p
Git Repo:
https://github.com/queso-fuego/uefi-dev
Repo state at the start of this video:
git clone --recurse-submodules https://github.com/queso-fuego/uefi-dev
cd uefi-dev
git checkout 50de8d4a551458b89c55748b62097a4a2d0704c5
Repo state at the end of (the next) video:
git checkout 3011ea7347916c753644c73b72b075639564964b
Join the Community Discord: https://discord.gg/yKm4T89QFn
Contact:
https://queso_fuego.srht.site/contact.html
If you feel like throwing money my way:
https://queso_fuego.srht.site/support.html
- Let me know if there's anything specific you'd like to see!
Questions about setup/software/etc.?
Check the FAQ: https://queso_fuego.srht.site/about.html
Editing/Misc. Notes:
OBS's default/recommended output format of fragmented mp4 didn't work in Davinci Resolve, either no audio loaded or moving clips around crashed it easily. Converting to regular mp4 did work, ffmpeg made that very easy and fast:
ffmpeg -i "in_video.mp4" -map 0 -c copy "out_video.mp4"
-map 0 keeps all video/audio channels and -c copy directly copies and does not convert/re-encode anything, this only changes the container.
I'll remember to check the "automatically remux to mp4" checkbox in OBS for the future, to not need to do all that.
I am thinking how to handle architecture specific code, as this and the next video are only for amd64/x86_64/IA-32e. I'm unfamiliar with paging and interrupts, among other things, on Aarch64, RISC-V, and other architectures. Conditional compilation via includes and make is fine, but I don't want it to be too clunky. Could also do separate source files with architecture specific make targets.
0:00:00 intro, changes since last video
0:04:51 looking at various wiki pages for x86_64 paging
0:08:52 start paging setup, function to allocate EFI mem map pages
0:28:22 map page function
0:47:11 unmap page function
0:50:05 identity map the efi memory map
0:58:12 set runtime virtual address map for runtime services
1:21:23 put kernel buffer at 4KB addresses to remap later (TBC)
Music credits:
"Pookatori and Friends" Kevin MacLeod (incompetech.com)
Licensed under Creative Commons: By Attribution 4.0 License
http://creativecommons.org/licenses/by/4.0/
#uefidev #cprogramming #paging