PPC405GP MMU: Problem in switching between Real and Translation modes

I had my BSP software for PowerPC PPC405GP processor working in Real mode just configured with 128 MB (0x00000000 – 0x07FFFFFF) Region as Cache enabled(ICCR=0x80000000; DCCR=0x80000000; DCRW=0x0; SGR=0x0; SU0R=0x0; SLER=0x0 ). To create both cacheable and non-cacheable memory regions within SDRAM(128 MB), it was necessary to implement Translation mode where TLB entries with smaller size pages can be created and different memory attributes can be set for each entry.

Enabling the Translation mode involves 1) Setting of the TLB entries 2) Setting the MSR register bits MSR[IR] and MSR[DR].

Howto set the TLB entries?

The following assembly routine gets Index, TLBHI(tag) portion and TLBLO(data) portion as input parameters and write the TLB entry.

    set_tlb_entry:           ; set_tlb_entry(tlb_index, TLBHI, TLBLO)
        sync
        tlbwe r4,r3,0        ; Write TLBHI
        tlbwe r5,r3,1        ; Write TLBLO
        isync

Using the above routine, I write the following entries.

/* TLB Index=0 */
/* EPN=0x00000000;Size=4MB;Valid=1;Endian=0;U0=0 */
/* RPN=0x00000000;EX=1;WR=1;ZSEL=0;W=0;I=0;M=0;G=0 */
/* This entry maps 0x0 physical address to same 0x0 virtual address;
    Enables the cache and grants Execute and Write permissions.
    (SDRAM for execution) */
set_tlb_entry(0, 0x00000340, 0x00000300);

/* TLB Index = 1 */
/* EPN=0x00400000;Size=4MB;Valid=1;Endian=0;U0=0 */
/* RPN=0x00400000;EX=0;WR=1;ZSEL=0;W=0;I=1;M=0;G=0 */
/* This entry maps 0x400000 physical address to same virtual address;
    Disables the cache and grants only Write permission.
    (SDRAM for IO Buffers) */
set_tlb_entry(1, 0x00400340, 0x00400104);

(Now, I have the vector table and execution code from 0x0 address.) And, it is time to enable the translation mode by setting the bits MSR[IR] and MSR[DR]. But, before doing this I must invalidate the instruction cache; flush and invalidate the data cache. Because, they have been used in different memory configuration by the Real mode. And, they are going to be used in different configuration by the TLB entries. Ok! Caches are flushed and invalidated. Then? I thought the Real mode settings can be reset to ICCR=0 and DCCR=0, since because once the Translation mode is enabled, Real mode settings become useless. So, what I did was I cleared all Real mode settings and enabled the Translation mode. But, when I ran my code at one location I got the exception thrown. Even I know the location, I am clueless why the exception thrown from that place. Because, the execution pass through that location only once at initialization. The execution went fine with the initialization. But, how that location is executed after that?

I found one strange thing that when I left the Real mode configuration as it is(Cache enabled for 128MB), the execution went fine. While thinking myself why the Real mode settings too needed while the Translation mode is enabled, I got the problem. When the Interrupt occurs at 0x500 or system call at 0xc00, the MSR[IR] and MSR[DR] bits are cleared. And, when these bits are cleared, processor automatically switches to Real mode. Till these MSR bits are restored, the access will be done through the Real mode. So, there is a possibility that same memory area has been accessed by Real mode and Translation mode with different cache attributes. So, the execution goes wrong.

My guess was right! During the system call at 0xc00, the context was stored at memory location 0x5000 when MSR[IR, DR] = 0, and they were stored/restored again when MSR[IR, DR]=1. When this continues, the data become unstable and causes undefined behaviour further.

So, what I did was, I set the same memory attributes for the memory page(first TLB entry) which contains 0x5000 address and the first Real mode region of  (0x0000 0000 – 0x07FFFFFF) so that at both Real mode and translation mode the memory region will be accessed with the same attributes.

So, my code sequence is as below:

1. At reset, MSR[IR, DR] is 0, and Real mode works with all cache inhibited.
2. At boot code, first 128 MB region is enabled with cache.
3. Now to switch to translation mode, Reset Real mode with all cache inhibited.
4. Invalidate Instruction cache.
5. Flush and Invalidate Data cache.
6. Write the TLB entries.
7. Set the Real mode (registers ICCR, DCCR, DCWR) with same attributes of the TLB entry corresponding to the context page.

This is also applicable to IBM PowerPC 405Fx (PPC405Fx core) and Xilinx PowerPC 405 (PPC405 core) processors.

(For any queries, please write in the feedback column)