技术领域Technical Field
本发明属于信息技术领域，具体涉及一种虚拟化下客户机调试方法、装置、介质及产品。The present invention belongs to the field of information technology, and in particular relates to a method, device, medium and product for debugging a client under virtualization.
背景技术Background Art
虚拟化技术的出现解决了传统硬件资源利用率低、灵活性不足、维护成本高等问题。通过虚拟化，可以将单个硬件划分为多个虚拟客户机，每个虚拟客户机都具有自己的操作系统、应用程序和资源。这种虚拟化技术使得在同一物理硬件上运行多个工作负载成为可能，从而提高了资源利用率、降低了成本，并简化了管理和部署。虚拟化技术在嵌入式系统和物联网领域也有越来越多的应用，这是由于随着嵌入式系统的功能越来越复杂，需要处理更多的任务和数据，嵌入式系统的设计变得越来越复杂。嵌入式虚拟化可以帮助解决这种复杂性，通过在单个嵌入式系统上运行多个虚拟化实例，将系统资源进行有效地管理和隔离。The emergence of virtualization technology solves the problems of low resource utilization, lack of flexibility, and high maintenance costs of traditional hardware. Through virtualization, a single hardware can be divided into multiple virtual clients, each with its own operating system, application, and resources. This virtualization technology makes it possible to run multiple workloads on the same physical hardware, thereby improving resource utilization, reducing costs, and simplifying management and deployment. Virtualization technology is also increasingly used in the fields of embedded systems and the Internet of Things. This is because as the functions of embedded systems become more and more complex and more tasks and data need to be processed, the design of embedded systems becomes more and more complex. Embedded virtualization can help solve this complexity by running multiple virtualization instances on a single embedded system to effectively manage and isolate system resources.
然而嵌入式系统不同于通用桌面或者服务器系统（ubuntu、centos等），它拥有更多的裁剪和定制需求。传统操作系统使用通用硬件（服务器）及通用软件，具有较好的兼容性和稳定性。嵌入式操作系统在资源限制、实时性要求、裁剪定制、功耗节能和实时性能等方面有着显著的不同。嵌入式操作系统在设计和实现时需要根据具体的应用场景进行优化和定制，以满足特定的需求和要求。这就导致嵌入式系统在虚拟化环境下的运行可能不是一帆风顺的。例如，通用操作系统一般无需修改就可以直接运行，而嵌入式系统在系统加载及启动初始化阶段就有可能因为各种原因而崩溃。However, embedded systems are different from general desktop or server systems (Ubuntu, CentOS, etc.), and they have more tailoring and customization requirements. Traditional operating systems use general hardware (servers) and general software, and have good compatibility and stability. Embedded operating systems are significantly different in terms of resource limitations, real-time requirements, tailoring and customization, power saving, and real-time performance. When designing and implementing embedded operating systems, they need to be optimized and customized according to specific application scenarios to meet specific needs and requirements. This results in the fact that the operation of embedded systems in virtualized environments may not be smooth sailing. For example, general operating systems can generally run directly without modification, while embedded systems may crash for various reasons during system loading and startup initialization.
总体来说，虚拟化中嵌入式系统客户机的调试难度较大。一般采取的方法是将修改好的系统在无虚拟化条件下进行调试，调试所有功能正常以后，再拿到虚拟化环境下运行。但虚拟化后的运行环境与真实硬件环境还是有差别的。虚拟化环境下，所有的硬件资源都是通过虚拟化软件虚拟出来的，而且虚拟出来的硬件数量可能与真实环境也有差别，这就带来系统兼容性的问题。当然这时还可以通过设备直通的方法，将串口、网口等设备透传给客户机，这样客户机操作系统就可以直接使用这些硬件作为调试口。例如，将硬件上的一个实际串口透传给操作系统作为系统的控制台，用户可以使用串口作为调试串口打印日志调试。在例如xen的虚拟化软件中，也有虚拟串口的工具，可为用户提供日志等功能。但这种调试手段必须是客户机启动到一定阶段以后才能生效。很多客户机载系统初始化阶段就会出现崩溃，而这时也没有任何日志输出，给调试带来了极大的难度。例如Linux系统在进行内存初始化阶段，这时串口还没有初始化好，这时崩溃的话没有任何输出，给用户带来极大的困难。In general, it is difficult to debug the embedded system client in virtualization. The general method is to debug the modified system without virtualization. After debugging all functions normally, run it in the virtualized environment. However, the virtualized operating environment is still different from the real hardware environment. In the virtualized environment, all hardware resources are virtualized by virtualization software, and the number of virtualized hardware may also be different from the real environment, which brings about the problem of system compatibility. Of course, at this time, you can also use the device pass-through method to pass serial ports, network ports and other devices to the client, so that the client operating system can directly use these hardware as debugging ports. For example, an actual serial port on the hardware is passed to the operating system as the system console, and the user can use the serial port as a debugging serial port to print log debugging. In virtualization software such as xen, there are also virtual serial port tools that can provide users with log and other functions. However, this debugging method must be effective after the client is started to a certain stage. Many clients will crash during the initialization stage of the onboard system, and there is no log output at this time, which brings great difficulty to debugging. For example, when the Linux system is in the memory initialization stage, the serial port has not been initialized yet. If it crashes at this time, there is no output, which brings great difficulties to users.
发明内容Summary of the invention
本发明的目的在于提供一种虚拟化下客户机调试方法、装置、介质及产品，解决虚拟化中嵌入式系统客户机难于调试的问题，尤其是客户机在启动阶段的bug和崩溃。通过增加调试手段，使用户可以定位到问题的关键从而提供解决思路。The purpose of the present invention is to provide a method, device, medium and product for debugging a client under virtualization, so as to solve the problem that it is difficult to debug a client of an embedded system in virtualization, especially the bugs and crashes of the client during the startup phase. By adding debugging means, the user can locate the key to the problem and provide a solution.
为了实现上述目的，本发明的技术方案是这样的：In order to achieve the above object, the technical solution of the present invention is as follows:
一种虚拟化下客户机调试方法，包括：A virtualized client debugging method, comprising:
基于xen Hypervisor虚拟机监控器，在dom0中运行调试进程，所述调试进程解析用户的调试命令，通过hvc指令陷入到hypervisor层；hypervisor层去采集虚拟客户机domu的关键数据结构的内存信息和CPU信息，并返回给调试进程。Based on the xen Hypervisor virtual machine monitor, a debugging process is run in dom0. The debugging process parses the user's debugging command and falls into the hypervisor layer through the hvc instruction; the hypervisor layer collects the memory information and CPU information of the key data structure of the virtual client domu and returns it to the debugging process.
进一步的，所述调试进程解析用户的调试命令时，加载虚拟客户机的内核符号表，得到内核中各个函数和变量的位置和用途，传递给hypervisor层。Furthermore, when the debugging process parses the debugging command of the user, it loads the kernel symbol table of the virtual client, obtains the location and purpose of each function and variable in the kernel, and passes it to the hypervisor layer.
进一步的，hypervisor层采集虚拟客户机domu的关键数据结构的内存信息的方法包括：Furthermore, the method for the hypervisor layer to collect memory information of key data structures of the virtual client domu includes:
S10、获取客户机的逻辑地址，将逻辑地址转换为实际的物理地址；S10, obtaining the logical address of the client, and converting the logical address into an actual physical address;
S11、将转换得到的物理地址映射到hypervisor层页表中，建立hypervisor层内部的地址映射；S11, mapping the converted physical address into the hypervisor layer page table, and establishing the address mapping inside the hypervisor layer;
S12、所述hypervisor层通过所述地址映射进行访问，将客户机操作系统的ringbuffer中的内核日志拷贝出来，进而返回给dom0的调试进程。S12, the hypervisor layer accesses through the address mapping, copies the kernel log in the ringbuffer of the client operating system, and then returns it to the debugging process of dom0.
更进一步的，步骤S10中逻辑地址转换为物理地址的方法包括：Furthermore, the method of converting the logical address into the physical address in step S10 includes:
S101、使用内存管理单元MMU的AT S12E1R/AT S12E1W指令，将虚拟地址写入到MMU的Address translation中进行地址翻译，再从物理地址寄存器par_el1中读出物理地址；若翻译失效则进入步骤S102；S101, using the AT S12E1R/AT S12E1W instruction of the memory management unit MMU, write the virtual address into the address translation of the MMU for address translation, and then read the physical address from the physical address register par_el1; if the translation fails, go to step S102;
S102、通过Linux操作系统内存内部的stage1页表转换，将逻辑地址转换为中间物理地址；转换过程包括：使用内存管理单元MMU的AT S1E1R/AT S1E1W指令将虚拟地址写入到MMU的Address translation中，再从物理地址寄存器par_el1中读出中间物理地址；如果直接使用硬件翻译失效，则使用软件的方式进行页表遍历，首先从TTBRn_EL1寄存器中读出stage1页表基地址，然后以虚拟地址的39到47位为索引，以L0基地址加上索引值得到第一级页表量，以此类推进行四级页表寻址，最终得到L3页表包含物理地址的47到12位，再加上逻辑地址的11到0位，最终得到48位的中间物理地址；进入步骤S103；S102, converting the logical address into an intermediate physical address through the stage1 page table conversion inside the Linux operating system memory; the conversion process includes: using the AT S1E1R/AT S1E1W instruction of the memory management unit MMU to write the virtual address into the address translation of the MMU, and then reading the intermediate physical address from the physical address register par_el1; if the direct use of hardware translation fails, the page table traversal is performed in software, firstly the stage1 page table base address is read from the TTBRn_EL1 register, and then the 39th to 47th bits of the virtual address are used as the index, and the first-level page table quantity is obtained by adding the L0 base address and the index value, and so on for the four-level page table addressing, and finally the L3 page table containing the 47th to 12th bits of the physical address is obtained, and then the 11th to 0th bits of the logical address are added, and finally the 48-bit intermediate physical address is obtained; then the process goes to step S103;
S103、通过hypervisor层内部stage2页表进行转换，将中间物理地址转换为实际的物理地址；转换过程包括：进行页表遍历，找到页表的基地址以后，从所述中间物理地址中取出对应位作为索引，然后进行四级页表寻址；以L0基地址加上索引值得到第一级页表量，以此类推进行四级页表寻址，最终得到L3页表包含物理地址的47到12位，再加上所述中间物理地址的11到0位，最终得到48位的物理地址。S103, converting the intermediate physical address into the actual physical address through the stage2 page table inside the hypervisor layer; the conversion process includes: performing page table traversal, after finding the base address of the page table, taking the corresponding bit from the intermediate physical address as an index, and then performing four-level page table addressing; adding the L0 base address to the index value to obtain the first-level page table quantity, and so on to perform four-level page table addressing, and finally obtaining the L3 page table containing 47 to 12 bits of the physical address, plus 11 to 0 bits of the intermediate physical address, and finally obtaining a 48-bit physical address.
进一步的，hypervisor层采集CPU信息的方法包括：Furthermore, the method for collecting CPU information at the hypervisor layer includes:
S201、对于客户机运行所在的物理CPU，hypervisor层通过当前CPU采集MMU/cache的寄存器，当前CPU的其他寄存器在发生异常前被保存在栈中，并通过参数的形式传递给hypervisor层的采集函数；所述当前CPU为运行调试进程的CPU；S201, for the physical CPU where the client runs, the hypervisor layer collects the registers of the MMU/cache through the current CPU, and other registers of the current CPU are saved in the stack before an exception occurs, and are passed to the collection function of the hypervisor layer in the form of parameters; the current CPU is the CPU running the debugging process;
S202、对于多核中的其它CPU，所述当前CPU向所述其它CPU发送一个sgi中断，其它CPU依次陷入到sgi的中断处理函数中，并在函数中判断是否要进行CPU信息采集，如果是的话，则采集流程与步骤S201相同；S202, for other CPUs in the multi-core, the current CPU sends an sgi interrupt to the other CPUs, and the other CPUs fall into the sgi interrupt processing function in turn, and determine whether to collect CPU information in the function. If yes, the collection process is the same as step S201;
S203、对于客户机的虚拟CPU，hypervisor层通过遍历物理CPU找到虚拟CPU的信息。S203: For the virtual CPU of the client, the hypervisor layer finds the information of the virtual CPU by traversing the physical CPU.
本发明另一方面还提出了一种虚拟化下客户机调试装置，包括：Another aspect of the present invention further provides a virtualized client debugging device, comprising:
调试模块：基于xen Hypervisor虚拟机监控器，在dom0中运行，解析用户的调试命令，通过hvc指令陷入到hypervisor层；Debug module: Based on the Xen Hypervisor virtual machine monitor, it runs in dom0, parses the user's debugging commands, and falls into the hypervisor layer through the hvc instruction;
内存采集模块：hypervisor层去采集虚拟客户机domu的关键数据结构的内存信息；Memory collection module: The hypervisor layer collects memory information of key data structures of virtual client domu;
CPU采集模块：hypervisor层去采集CPU信息；CPU collection module: The hypervisor layer collects CPU information;
交互模块：将内存采集模块和CPU采集模块采集的信息返回给调试模块。Interaction module: returns the information collected by the memory acquisition module and the CPU acquisition module to the debugging module.
进一步的，所述内存采集模块包括：Furthermore, the memory acquisition module includes:
转换子模块：获取客户机的逻辑地址，将逻辑地址转换为实际的物理地址；Conversion submodule: obtains the logical address of the client and converts the logical address into the actual physical address;
映射子模块：将转换得到的物理地址映射到hypervisor层页表中，建立hypervisor层内部的地址映射；Mapping submodule: maps the converted physical address into the hypervisor layer page table and establishes the address mapping inside the hypervisor layer;
拷贝子模块：所述hypervisor层通过所述地址映射进行访问，将客户机操作系统的ring buffer中的内核日志拷贝出来，进而返回给dom0的调试进程。Copy submodule: The hypervisor layer accesses through the address mapping, copies the kernel log in the ring buffer of the client operating system, and then returns it to the debugging process of dom0.
更进一步的，所述转换子模块包括：Furthermore, the conversion submodule includes:
直接转换单元：使用内存管理单元MMU的AT S12E1R/AT S12E1W指令，将虚拟地址写入到MMU的Address translation中进行地址翻译，再从物理地址寄存器par_el1中读出物理地址；若翻译失效则进入stage1转换单元；Direct translation unit: Use the AT S12E1R/AT S12E1W instruction of the memory management unit MMU to write the virtual address into the MMU's Address translation for address translation, and then read the physical address from the physical address register par_el1; if the translation fails, enter the stage1 translation unit;
stage1转换单元：通过Linux操作系统内存内部的stage1页表转换，将逻辑地址转换为中间物理地址；转换过程包括：使用内存管理单元MMU的AT S1E1R/AT S1E1W指令将虚拟地址写入到MMU的Address translation中，再从物理地址寄存器par_el1中读出中间物理地址；如果直接使用硬件翻译失效，则使用软件的方式进行页表遍历，首先从TTBRn_EL1寄存器中读出stage1页表基地址，然后以虚拟地址的39到47位为索引，以L0基地址加上索引值得到第一级页表量，以此类推进行四级页表寻址，最终得到L3页表包含物理地址的47到12位，再加上逻辑地址的11到0位，最终得到48位的中间物理地址；进入stage2转换单元；Stage 1 translation unit: The logical address is translated into an intermediate physical address through the stage 1 page table translation inside the Linux operating system memory. The translation process includes: using the AT S1E1R/AT S1E1W instruction of the memory management unit MMU to write the virtual address into the address translation of the MMU, and then reading the intermediate physical address from the physical address register par_el1. If the direct use of hardware translation fails, the page table traversal is performed in software. First, the stage 1 page table base address is read from the TTBRn_EL1 register, and then the 39 to 47 bits of the virtual address are used as the index, and the first-level page table quantity is obtained by adding the L0 base address and the index value. The four-level page table addressing is performed in this way, and finally the L3 page table containing the 47 to 12 bits of the physical address is obtained, and then the 11 to 0 bits of the logical address are added, and finally the 48-bit intermediate physical address is obtained. Enter the stage 2 translation unit.
stage2转换单元：通过hypervisor层内部stage2页表进行转换，将中间物理地址转换为实际的物理地址；转换过程包括：进行页表遍历，找到页表的基地址以后，从所述中间物理地址中取出对应位作为索引，然后进行四级页表寻址；以L0基地址加上索引值得到第一级页表量，以此类推进行四级页表寻址，最终得到L3页表包含物理地址的47到12位，再加上所述中间物理地址的11到0位，最终得到48位的物理地址。Stage2 conversion unit: converts the intermediate physical address into the actual physical address through the stage2 page table inside the hypervisor layer; the conversion process includes: traversing the page table, finding the base address of the page table, taking the corresponding bit from the intermediate physical address as an index, and then performing four-level page table addressing; adding the L0 base address to the index value to obtain the first-level page table quantity, and so on for four-level page table addressing, and finally obtaining the L3 page table containing 47 to 12 bits of the physical address, plus 11 to 0 bits of the intermediate physical address, and finally obtaining a 48-bit physical address.
进一步的，所述调试模块包括加载单元，在解析用户的调试命令时，加载虚拟客户机的内核符号表，得到内核中各个函数和变量的位置和用途，传递给hypervisor层。Furthermore, the debugging module includes a loading unit, which loads the kernel symbol table of the virtual client when parsing the user's debugging command, obtains the location and purpose of each function and variable in the kernel, and passes it to the hypervisor layer.
进一步的，所述CPU采集模块包括：Furthermore, the CPU acquisition module includes:
物理CPU采集单元：对于客户机运行所在的物理CPU，hypervisor层通过当前CPU采集MMU/cache的寄存器，当前CPU的其他寄存器在发生异常前被保存在栈中，并通过参数的形式传递给hypervisor层的采集函数；所述当前CPU为运行调试模块的CPU；Physical CPU acquisition unit: For the physical CPU where the client runs, the hypervisor layer acquires the registers of the MMU/cache through the current CPU. The other registers of the current CPU are saved in the stack before an exception occurs and are passed to the acquisition function of the hypervisor layer in the form of parameters. The current CPU is the CPU running the debugging module.
多核CPU采集单元：对于多核中的其它CPU，所述当前CPU向所述其它CPU发送一个sgi中断，其它CPU依次陷入到sgi的中断处理函数中，并在函数中判断是否要进行CPU信息采集，如果是的话，则采集流程与物理CPU采集单元相同；Multi-core CPU collection unit: For other CPUs in the multi-core, the current CPU sends an sgi interrupt to the other CPUs, and the other CPUs fall into the sgi interrupt processing function in turn, and determine whether to collect CPU information in the function. If so, the collection process is the same as that of the physical CPU collection unit;
虚拟CPU采集单元：对于客户机的虚拟CPU，hypervisor层通过遍历物理CPU找到虚拟CPU的信息。Virtual CPU collection unit: For the virtual CPU of the client, the hypervisor layer finds the information of the virtual CPU by traversing the physical CPU.
本发明还提供了一种计算机可读存储介质，所述存储介质存储有计算机程序，所述计算机程序用于执行上述的虚拟化下客户机调试方法。The present invention also provides a computer-readable storage medium, wherein the storage medium stores a computer program, and the computer program is used to execute the above-mentioned virtualized client debugging method.
本发明还提供了一种计算机程序产品，包括计算机程序，所述计算机程序被处理器执行时实现上述的虚拟化下客户机调试方法。The present invention also provides a computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the computer program implements the above-mentioned method for debugging a client under virtualization.
与现有技术相比，本发明具有如下的有益效果：Compared with the prior art, the present invention has the following beneficial effects:
本发明方案大大降低了系统调试的成本，在不需要额外增加硬件设备的情况下就可以采集到系统信息。同时，增加了系统调试的应用场景。对于已经成型的产品或者没有JTAG调试口的硬件，都可以进行系统调试。基于本发明思路，可以扩展调试模块的功能，进而采集到等多的信息以满足开发人员的需求。The solution of the present invention greatly reduces the cost of system debugging, and system information can be collected without adding additional hardware equipment. At the same time, the application scenarios of system debugging are increased. For finished products or hardware without JTAG debugging ports, system debugging can be performed. Based on the idea of the present invention, the function of the debugging module can be expanded, and more information can be collected to meet the needs of developers.
附图说明BRIEF DESCRIPTION OF THE DRAWINGS
图1是本发明实施例的硬件与软件实现系统调试原理图对比。FIG. 1 is a comparison of the hardware and software system debugging principle diagrams of an embodiment of the present invention.
图2是本发明实施例中虚拟化下客户机地址到物理地址转换示意图。FIG. 2 is a schematic diagram of conversion from a client address to a physical address under virtualization in an embodiment of the present invention.
图3是本发明实施例中软件方式进行页表遍历原理图。FIG. 3 is a schematic diagram showing a principle of performing page table traversal in a software manner according to an embodiment of the present invention.
图4是虚拟化下客户机地址到物理地址转换流程图。FIG. 4 is a flow chart of client address to physical address conversion under virtualization.
具体实施方式DETAILED DESCRIPTION
需要说明的是，在不冲突的情况下，本发明中的实施例及实施例中的特征可以相互组合。It should be noted that, in the absence of conflict, the embodiments of the present invention and the features in the embodiments may be combined with each other.
芯片调试中的Jlink或者trace32调试仿真器，无需通过串口，而是通过JTAG接口就可以直接访问芯片内部总线以及CPU内部寄存器等内容。调试器独立于芯片，站在一个监管和控制的角度去采集芯片内部的信息来提供给开发者。本发明的设计思想与调试器类似，但不同的是使用软件来实现这些工作，如图1所示。因为通常这类调试器的成本都比较昂贵，大部分用户可能并不具备这类设备。另外，开发人员所调试的硬件并不都具有JTAG接口，这类接口通常只在芯片开发的初期使用。大部分产品在成型以后，都设计为无JTAG接口的电路。The Jlink or trace32 debugging simulator in chip debugging can directly access the chip's internal bus and CPU internal registers through the JTAG interface instead of the serial port. The debugger is independent of the chip, and collects information inside the chip from a supervisory and control perspective to provide it to developers. The design concept of the present invention is similar to that of the debugger, but the difference is that software is used to implement these tasks, as shown in Figure 1. Because the cost of such debuggers is usually relatively expensive, most users may not have such equipment. In addition, not all hardware debugged by developers has a JTAG interface, and such interfaces are usually only used in the early stages of chip development. After most products are formed, they are designed as circuits without a JTAG interface.
本发明使用hypervisor来充当trace32的角色，调试的目标是客户机，而客户机是运行在hypervisor层之上。hypervisor层对客户机来说拥有更多的权限，而且独立于客户机，正好可以站在一个监管和控制的角度去采集芯片内部的信息。这些芯片内部信息对于开发者来说具有极大的价值。The present invention uses the hypervisor to play the role of trace32. The debugging target is the client, and the client runs on the hypervisor layer. The hypervisor layer has more permissions for the client and is independent of the client. It can collect information inside the chip from a supervisory and control perspective. This chip internal information is of great value to developers.
本发明基于ARMv8架构，虚拟化软件Hypervisor使用Xen 4.17.0版本。The present invention is based on the ARMv8 architecture, and the virtualization software Hypervisor uses the Xen 4.17.0 version.
所述xen Hypervisor软件的组成分为三部分：The xen Hypervisor software consists of three parts:
(1) 硬件层之上的hypervisor层：负责直接驱动CPU和Memory这些基础硬件，为其它所有虚拟机提供CPU、内存、Interrupt中断管理，并且还提供了HyperCall的调用；(1) The hypervisor layer above the hardware layer: It is responsible for directly driving the basic hardware such as CPU and Memory, providing CPU, memory, and interrupt management for all other virtual machines, and also providing HyperCall calls;
(2) dom0：特权虚拟机，有整个虚拟化环境的访问权，并负责创建用户级虚拟机，并为其分配I/O设备资源；(2) dom0: a privileged virtual machine that has access to the entire virtualization environment and is responsible for creating user-level virtual machines and allocating I/O device resources to them;
(3) domu：用户级虚拟机，是实际提供给用户使用的虚拟机，也是本发明中虚拟客户机。(3) domu: a user-level virtual machine, which is a virtual machine actually provided to users for use and is also the virtual client in the present invention.
调试的目标具体来说是虚拟客户机的Linux内核程序，因为虚拟客户机崩溃最难解决的问题是出现在内核层，尤其是早期系统初始化时。当系统由内核进入到文件系统以后，用户就有等多的手段进行用户层程序的调试。当系统出现宕机崩溃，同时也没有任何日志输出。最需要关注的就是目前CPU的状态（包括PC指针所处的位置、栈帧调用以及相关寄存器）、内存信息（内存中的关键数据结构）等。本施例主要从以上两个方面对宕机以后的客户机内部信息进行采集。如图1所示，虚拟客户机domu所有的硬件资源都是由hypervisor层提供的，有些是通过虚拟化模拟而成（如VCPU、中断、部分外设等），有些是由hypervisor层通过页表映射的方式直接提供（如内存、部分外设等）。所以这些信息都可以被hypervisor层追踪到，这是使用hypervisor层模拟仿真器调试的基本原理。Specifically, the debugging target is the Linux kernel program of the virtual client, because the most difficult problem to solve when the virtual client crashes occurs at the kernel layer, especially during the initialization of the early system. After the system enters the file system from the kernel, the user has many means to debug the user-level program. When the system crashes, there is no log output at the same time. The most important thing to pay attention to is the current CPU status (including the location of the PC pointer, stack frame calls and related registers), memory information (key data structures in the memory), etc. This example mainly collects the internal information of the client after the crash from the above two aspects. As shown in Figure 1, all the hardware resources of the virtual client domu are provided by the hypervisor layer, some of which are simulated by virtualization (such as VCPU, interrupts, some peripherals, etc.), and some are directly provided by the hypervisor layer through page table mapping (such as memory, some peripherals, etc.). Therefore, all this information can be tracked by the hypervisor layer, which is the basic principle of using the hypervisor layer simulation simulator for debugging.
1、使用dom0作为调试主机：1. Use dom0 as the debugging host:
在xen的架构中，dom0是一个特别的虚拟机，是xen启动以后启动的第一个虚拟机，拥有最高的权限，负责管理其它客户机，所以也可以称为管理域。所以这里使用dom0作为调试主机，在dom0中启动一个调试进程（图1中的调试软件）。调试进程是用户的前端接口，负责解析用户的调试命令，最终通过hvc指令陷入到hypervisor层中。hypervisor层中去采集客户机的硬件和软件资源，并经过计算处理以后得到用户想要的数据返回给调试软件。最终用户在调试软件中查看调试结果。In the architecture of Xen, dom0 is a special virtual machine. It is the first virtual machine started after Xen is started. It has the highest authority and is responsible for managing other clients, so it can also be called a management domain. Therefore, dom0 is used as the debugging host here, and a debugging process (debugging software in Figure 1) is started in dom0. The debugging process is the user's front-end interface, responsible for parsing the user's debugging commands, and finally falls into the hypervisor layer through the hvc instruction. The hypervisor layer collects the hardware and software resources of the client, and after calculation and processing, obtains the data that the user wants and returns it to the debugging software. The final user views the debugging results in the debugging software.
2、在调试软件中加载客户机的内核符号表：2. Load the client's kernel symbol table in the debugging software:
内核符号表是一个存储着内核中各种函数、变量和其他符号信息的数据库，提供了对内核代码的索引和访问。符号表通常存储在编译后的内核映像中，并且可以通过调试工具或特殊的命令来查看和操作。调试软件使用gdb以及objdump工具得到内核中各个函数和变量的位置和用途，从而更容易地进行调试、优化和扩展。The kernel symbol table is a database that stores various functions, variables, and other symbolic information in the kernel, providing indexing and access to the kernel code. The symbol table is usually stored in the compiled kernel image and can be viewed and manipulated through debugging tools or special commands. The debugging software uses gdb and objdump tools to obtain the location and purpose of each function and variable in the kernel, making it easier to debug, optimize, and expand.
3、关键数据结构的内存信息采集：3. Memory information collection of key data structures:
既然符号表中包含了系统调试的重要信息，接下来的工作就是从客户机的内存中读取这些信息。本实施例以内核日志的采集为例进行说明。Since the symbol table contains important information for system debugging, the next task is to read this information from the client's memory. This embodiment is described by taking the collection of kernel logs as an example.
内核日志是系统调试的重要手段，但是如果系统是在启动的早期阶段崩溃，或者系统没有有效的串口硬件，就会导致系统没有输出。开发人员也无法判断内核是在那个位置崩溃的。内核日志的核心是一个叫做log buffer的循环缓冲区（ring buffer），printk作为生产者将消息存入该缓冲区，log服务模块作为消费者可从log buffer中读取消息。所谓log服务模块可以是串口、dmesg等输出端口。内存信息被解析之前，系统只映射了内核镜像所在位置内存的虚拟地址，memblock和buddy都没有初始化，因此此时无法通过动态方式分配内存。为了支持printk，此时内核可以通过全局变量方式定义一个__log_buf[__LOG_BUF_LEN]的静态数组，并将数组地址赋值给log_buf（ring buffer指针）。待内存初始化完成以后，系统调用memblock内存管理器为ring buffer重新分配动态内存。Kernel log is an important means of system debugging, but if the system crashes in the early stage of startup, or the system does not have effective serial port hardware, the system will have no output. Developers cannot determine where the kernel crashed. The core of the kernel log is a ring buffer called log buffer. Printk, as a producer, stores messages in the buffer, and the log service module, as a consumer, can read messages from the log buffer. The so-called log service module can be an output port such as a serial port or dmesg. Before the memory information is parsed, the system only maps the virtual address of the memory where the kernel image is located. Neither memblock nor buddy are initialized, so memory cannot be allocated dynamically at this time. In order to support printk, the kernel can define a static array of __log_buf[__LOG_BUF_LEN] in the form of global variables, and assign the array address to log_buf (ring buffer pointer). After the memory initialization is completed, the system calls the memblock memory manager to reallocate dynamic memory for the ring buffer.
首先通过“ objdump -t vmlinux | grep log_buf”，可以得到log_buf的地址为ffff000008fe1c48，这是一个客户机的逻辑地址（gva）,如图2所示。gva到实际的物理地址需要经过两个阶段的转换，第一阶段stage1是Linux操作系统内存内部的stage1页表（页表基地址保存在TTBRn_EL1寄存器）转换，将gva转换为中间物理地址IPA。第二阶段stage2由hypervisor层内部stage2页表（页表基地址保存在VTTBR0_EL2）进行转换，将IPA转换为最终的物理地址PA。不同于客户机的内存映射，hypervisor层想要访问物理地址也需要经过hypervisor层页表（页表基地址保存在TTBR0_EL2）。First, through "objdump -t vmlinux | grep log_buf", we can get the address of log_buf as ffff000008fe1c48, which is a logical address (gva) of the client, as shown in Figure 2. The conversion from gva to the actual physical address requires two stages. The first stage, stage1, is the conversion of the stage1 page table inside the Linux operating system memory (the page table base address is stored in the TTBRn_EL1 register), which converts gva to the intermediate physical address IPA. The second stage, stage2, is converted by the stage2 page table inside the hypervisor layer (the page table base address is stored in VTTBR0_EL2), which converts IPA to the final physical address PA. Different from the memory mapping of the client, the hypervisor layer also needs to go through the hypervisor layer page table (the page table base address is stored in TTBR0_EL2) to access the physical address.
如图4所示，想要将客户机虚拟地址转换为物理地址最直接的办法是使用MMU的ATS12E1R/AT S12E1W指令，将虚拟地址写入到MMU的Address translation中，再从par_el1中读出的物理地址。由于MMU已经开启了EL2模式，所以这种指令翻译直接将gva转换为了PA。然而由于页表条目失效等原因，有可能导致指令执行错误。这时候就要退而求其次，将stage1和stage2分别转换。首先将gva转换为IPA，同理可以使用MMU的AT S1E1R/AT S1E1W指令将虚拟地址写入到MMU的Address translation中，再从par_el1中读出的物理地址。这时读到的是IPA地址。如果直接使用硬件翻译失效，那需要考虑使用软件的方式进行页表遍历。如图3所示，首先从TTBRn_EL1寄存器中读出stage1页表基地址，该地址指向L0页表。然后以虚拟地址的39到47位为索引，以L0基地址加上索引值得到第一级页表量。以此类推进行四级页表寻址，最终得到L3页表包含了物理地址的47到12位，再加上VA的11到0位，最终得到48位的物理地址也就是stage1的转换地址IPA。然后再进行stage2转换。stage2的转换与stage1不同的地方是，首先它不能使用硬件翻译，因为没有只进行stage2转换的硬件指令。所以只能使用软件的方式进行页表遍历。xen为每个客户机都模拟了一个内存空间，其中页表的基地址都保存在p2m_domain结构中。找到基地址以后，从IPA的地址中取出对应位作为索引，然后进行四级页表寻址。其过程与上述stage1的页表遍历相同，这里不再赘述。得到PA以后还需要将该地址映射到hypervisor层页表中，建立hypervisor层内部的地址映射（如图2所示）。这样hypervisor层内部就可以访问这块内存，从而采集到相应的内容。As shown in Figure 4, the most direct way to convert the client virtual address to the physical address is to use the MMU's ATS12E1R/AT S12E1W instructions to write the virtual address into the MMU's Address translation, and then read the physical address from par_el1. Since the MMU has already turned on the EL2 mode, this instruction translation directly converts gva to PA. However, due to reasons such as invalid page table entries, it may cause instruction execution errors. At this time, we have to settle for the next best thing and convert stage1 and stage2 separately. First, convert gva to IPA. Similarly, the MMU's AT S1E1R/AT S1E1W instructions can be used to write the virtual address into the MMU's Address translation, and then read the physical address from par_el1. At this time, the IPA address is read. If direct hardware translation fails, it is necessary to consider using software to traverse the page table. As shown in Figure 3, first read the stage1 page table base address from the TTBRn_EL1 register, which points to the L0 page table. Then, using bits 39 to 47 of the virtual address as the index, the L0 base address plus the index value is used to obtain the first-level page table quantity. This is followed by four-level page table addressing, and the final L3 page table contains bits 47 to 12 of the physical address, plus bits 11 to 0 of VA, and finally a 48-bit physical address, which is the stage1 conversion address IPA. Then stage2 conversion is performed. The difference between stage2 conversion and stage1 is that, first of all, it cannot use hardware translation, because there is no hardware instruction that only performs stage2 conversion. Therefore, page table traversal can only be performed in software. Xen simulates a memory space for each client, in which the base address of the page table is stored in the p2m_domain structure. After finding the base address, the corresponding bit is taken from the address of IPA as an index, and then four-level page table addressing is performed. The process is the same as the above stage1 page table traversal, and will not be repeated here. After obtaining PA, it is also necessary to map the address to the hypervisor layer page table and establish the address mapping inside the hypervisor layer (as shown in Figure 2). In this way, the hypervisor layer can access this memory and collect the corresponding content.
通过上述方法，最终将客户机操作系统的ring buffer中的内核日志拷贝出来，进而返回给dom0的调试软件。这样即使客户机内核在早期发生崩溃，也能取出日志从而定位到发生崩溃的位置。除此之外也可以用相同的方法取出客户机页表、中断、内存信息等等，取出内容可以通过调试软件控制输入和输出。Through the above method, the kernel log in the ring buffer of the client operating system is finally copied out and then returned to the debugging software of dom0. In this way, even if the client kernel crashes at an early stage, the log can be retrieved to locate the crash location. In addition, the same method can be used to retrieve the client page table, interrupt, memory information, etc. The retrieved content can be controlled by the debugging software for input and output.
4、CPU信息采集：4. CPU information collection:
CPU寄存器的采集分为两类，一是客户机运行所在的物理CPU，二是客户机的虚拟CPU（VCPU）。当调试软件向hypervisor层发送CPU的查询命令以后，执行CPU信息采集程序，当前CPU（运行调试软件的CPU）首先通过arm64汇编指令采集MMU cache等相关硬件的寄存器。而CPU的其它寄存器在发生异常前都被保存在了栈中，并通过参数的形式传递给hypervisor层的采集函数。hypervisor层可以从栈中读取并保存输出给调试软件。对于多核中的其它CPU，则需要运行调试软件的CPU向它们发送一个sgi中断。其它CPU依次陷入到sgi的中断处理函数中，并在函数中判断是否要进行CPU信息采集。如果是的话，则执行CPU信息采集程序，其流程与上述过程相同。VCPU的信息采集与物理CPU不同的地方是，VCPU由软件模拟而成，所以不需要操作实际的硬件。VCPU的信息保存在VCPU的数据结构中，而VCPU数据结构以队列的形式挂在物理CPU上，所以最终可以通过遍历CPU找到VCPU的信息，并输出给调试软件。The collection of CPU registers is divided into two categories, one is the physical CPU where the client runs, and the other is the virtual CPU (VCPU) of the client. After the debugging software sends a CPU query command to the hypervisor layer, the CPU information collection program is executed. The current CPU (the CPU running the debugging software) first collects the registers of related hardware such as MMU cache through arm64 assembly instructions. The other registers of the CPU are saved in the stack before an exception occurs, and are passed to the collection function of the hypervisor layer in the form of parameters. The hypervisor layer can read from the stack and save the output to the debugging software. For other CPUs in the multi-core, the CPU running the debugging software needs to send them an sgi interrupt. Other CPUs fall into the interrupt processing function of sgi in turn, and determine whether to collect CPU information in the function. If so, the CPU information collection program is executed, and its process is the same as the above process. The difference between VCPU information collection and physical CPU is that VCPU is simulated by software, so there is no need to operate actual hardware. The VCPU information is stored in the VCPU data structure, and the VCPU data structure is hung on the physical CPU in the form of a queue, so the VCPU information can be found by traversing the CPU and output to the debugging software.
以上所述实施例仅是本发明的优选实施方式，只是用于帮助理解本申请的方法及其核心思想，本发明的保护范围并不仅局限于上述实施例，凡属于本发明思路下的技术方案均属于本发明的保护范围。应当指出，对于本技术领域的普通技术人员来说，在不脱离本发明原理前提下的若干改进和润饰，这些改进和润饰也应视为本发明的保护范围。The above-mentioned embodiments are only preferred implementations of the present invention, and are only used to help understand the method and core ideas of the present application. The protection scope of the present invention is not limited to the above-mentioned embodiments. All technical solutions under the idea of the present invention belong to the protection scope of the present invention. It should be pointed out that for ordinary technicians in this technical field, some improvements and modifications without departing from the principle of the present invention should also be regarded as the protection scope of the present invention.
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202411177921.5ACN118708498B (en) | 2024-08-27 | 2024-08-27 | A method, device, medium and product for debugging a client under virtualization |
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202411177921.5ACN118708498B (en) | 2024-08-27 | 2024-08-27 | A method, device, medium and product for debugging a client under virtualization |
| Publication Number | Publication Date |
|---|---|
| CN118708498A CN118708498A (en) | 2024-09-27 |
| CN118708498Btrue CN118708498B (en) | 2024-11-05 |
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202411177921.5AActiveCN118708498B (en) | 2024-08-27 | 2024-08-27 | A method, device, medium and product for debugging a client under virtualization |
| Country | Link |
|---|---|
| CN (1) | CN118708498B (en) |
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107797895A (en)* | 2017-05-08 | 2018-03-13 | 中国人民解放军国防科学技术大学 | A kind of secure virtual machine monitoring method and system |
| CN113050483A (en)* | 2021-03-11 | 2021-06-29 | 深圳市阿丹能量信息技术有限公司 | Cabin domain controller system based on i.MX8 platform and Xen technology and application method |
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| KR101454837B1 (en)* | 2013-04-22 | 2014-10-28 | 한국인터넷진흥원 | Hypervisor security API module and hypervisor-based virtual network intrusion prevention system |
| CN116340035A (en)* | 2023-02-14 | 2023-06-27 | 麒麟软件有限公司 | Method for debugging debug jailhouse by dynamically setting log level |
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107797895A (en)* | 2017-05-08 | 2018-03-13 | 中国人民解放军国防科学技术大学 | A kind of secure virtual machine monitoring method and system |
| CN113050483A (en)* | 2021-03-11 | 2021-06-29 | 深圳市阿丹能量信息技术有限公司 | Cabin domain controller system based on i.MX8 platform and Xen technology and application method |
| Publication number | Publication date |
|---|---|
| CN118708498A (en) | 2024-09-27 |
| Publication | Publication Date | Title |
|---|---|---|
| CN113806006B (en) | A method and device for processing exceptions or interrupts under a heterogeneous instruction set architecture | |
| Goldberg | Survey of virtual machine research | |
| Chisnall | The definitive guide to the xen hypervisor | |
| JP5068778B2 (en) | Method and system for managing machine state in virtual machine operation | |
| US7434003B2 (en) | Efficient operating system operation on a hypervisor | |
| JP7538950B2 (en) | COMPUTER DEVICE, EXCEPTION PROCESSING METHOD, AND INTERRUPT PROCESSING METHOD - Patent application | |
| US8887139B2 (en) | Virtual system and method of analyzing operation of virtual system | |
| JP2004110809A (en) | Method and system for multiprocessor emulation on multiprocessor host system | |
| JP7592858B2 (en) | Chip system, virtual interrupt processing method and corresponding device - Patents.com | |
| CN101751284A (en) | I/O resource scheduling method for distributed virtual machine monitor | |
| CN114327777B (en) | Method and device for determining global page directory, electronic equipment and storage medium | |
| AU2006200310A1 (en) | Systems and methods for multi-level intercept processing in a virtual machine environment | |
| US11693722B2 (en) | Fast memory mapped IO support by register switch | |
| US7499057B2 (en) | Address translation in an integrated graphics environment | |
| EP2941694B1 (en) | Capability based device driver framework | |
| WO2013088818A1 (en) | Virtual computer system, virtualization mechanism, and data management method | |
| WO2014107540A1 (en) | Software interface for a hardware device | |
| US20050246708A1 (en) | Method of assigning virtual process identifier to process within process domain | |
| CN117573292B (en) | Method for Xen running general RTOS virtual machine | |
| CN118708498B (en) | A method, device, medium and product for debugging a client under virtualization | |
| EP2941695B1 (en) | High throughput low latency user mode drivers implemented in managed code | |
| JP2003067206A (en) | Method of dynamically assigning device in lpar system | |
| JP2523502B2 (en) | Control system of recursive virtual machine | |
| CN118519859B (en) | Driver information acquisition method, device and storage medium | |
| Martin | VIRTUAL MACHINE INTROSPECTION TOOL DESIGN ANALYSIS |
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |