Monday, 25 September 2017 17:56

The Case of Duplicate Firmware Objects in BCD

Written by
Rate this item
(5 votes)

image

A few months ago a customer complained that on a Dell Optiplex 7040 MiniTower the boot menu contained multiple entries for the "Windows Boot Manager". Given that we were in the process of deploying Windows 10 client and the importance of the customer, I immediately started troubleshooting. This particular case is especially interesting because it might affect a large number of users and the vendor was not aware of the issue.

My first step was to run bcdedit /enum firmware command to enumerate the firmware namespace objects in BCD. The output showed entries for local devices, such as a CD-ROM and NIC as well as one valid "Windows Boot Manager" entry. In addition, a large number of orphaned "Windows Boot Manager" entries was present in the system boot configuration data (BCD).

The following example is snippet of the output from one of the affected systems:

Firmware Boot Manager
---------------------
identifier              {fwbootmgr}
displayorder            {bootmgr}
                        {83222746-075c-11e6-b382-64006a688434}
                        {83222757-075c-11e6-b382-64006a688434}
                        {83222755-075c-11e6-b382-64006a688434}
                        {83222754-075c-11e6-b382-64006a688434}
                        {83222753-075c-11e6-b382-64006a688434}
                        {83222752-075c-11e6-b382-64006a688434}
                        {83222751-075c-11e6-b382-64006a688434}
                        {83222750-075c-11e6-b382-64006a688434}
                        {8322274f-075c-11e6-b382-64006a688434}
                        {8322274e-075c-11e6-b382-64006a688434}
                        {8322274d-075c-11e6-b382-64006a688434}
                        {8322274c-075c-11e6-b382-64006a688434}
                        {8322274b-075c-11e6-b382-64006a688434}
                        {8322274a-075c-11e6-b382-64006a688434}
                        {83222741-075c-11e6-b382-64006a688434}
                        {83222742-075c-11e6-b382-64006a688434}
                        {83222743-075c-11e6-b382-64006a688434}
                        {83222744-075c-11e6-b382-64006a688434}
                        {83222745-075c-11e6-b382-64006a688434}
                        {83222747-075c-11e6-b382-64006a688434}
                        {83222748-075c-11e6-b382-64006a688434}
                        {83222749-075c-11e6-b382-64006a688434}
timeout                 2

Windows-Start-Manager
---------------------
identifier              {bootmgr}
device                  partition=\Device\HarddiskVolume1
path                    \EFI\Microsoft\Boot\bootmgfw.efi
description             Windows Boot Manager
locale                  de-DE
inherit                 {globalsettings}
badmemoryaccess         Yes
default                 {current}
resumeobject            {8322273f-075c-11e6-b382-64006a688434}
displayorder            {current}
toolsdisplayorder       {memdiag}
timeout                 30

Firmware Application (101fffff)
----------------------------
identifier              {83222741-075c-11e6-b382-64006a688434}
description             Windows Boot Manager
badmemoryaccess         Yes

Firmware Application (101fffff)
----------------------------
identifier              {83222742-075c-11e6-b382-64006a688434}
description             Windows Boot Manager
badmemoryaccess         Yes

Firmware Application (101fffff)
----------------------------
identifier              {83222748-075c-11e6-b382-64006a688434}
description             Onboard NIC(IPV4)
badmemoryaccess         Yes

This particular system was used to test Windows 10 image, so it was being reinstalled fairly often. I suspected that a duplicate "Windows Boot Manager" entry was created during OSD. Multiple installations resulted in many duplicate entries.  I likely could have worked around the problem for the time being by setting boot menu timeout to 0 resulting in the default operating system starting instantly, but knew that if I didn’t determine the root cause and address it, the customer will run into the issue at some point. In any case, stooping to that kind of troubleshooting hack is beneath my dignity. I therefore set out to investigate

From past experience I knew that on some Unified Extensible Firmware Interface (UEFI)-based systems, the firmware creates non-volatile random access memory (NVRAM) entries for local devices when the computer boots. When MDT tries to create boot loader entries it executes BCDBoot, followed by bcdedit:

\\MDT01\DeploymentShare$\Tools\X86\BCDBoot.exe C:\Windows /l de-DE  /s V:
X:\WINDOWS\SYSTEM32\bcdedit.exe /Store "V:\boot\bcd" /timeout 0

When bcdedit opens the BCD, it compares entries in NVRAM with entries in BCD. Entries in NVRAM that were created by the firmware that do not exist in BCD are added to the system BCD. When bcdedit closes the system BCD, any boot manager entries in BCD that are not in NVRAM are added to NVRAM. This behavior might result in multiple firmware entries for "Windows Boot Manager" with different GUIDs on some devices that are similar to the following example:

Firmware Application (101fffff)
----------------------------
identifier              {83222741-075c-11e6-b382-64006a688434}
description             Windows Boot Manager
badmemoryaccess         Yes

Firmware Application (101fffff)
----------------------------
identifier              {83222742-075c-11e6-b382-64006a688434}
description             Windows Boot Manager
badmemoryaccess         Yes

The first thing I do when I suspect that some kind of model specific problem is to contact the product group. Unfortunately, none of my contacts were aware of the issue. Adding to the mystery only one particular Dell model was affected - even the sister model, Dell Optiplex 7040 Micro, was not showing the systems. I was at a dead end. I couldn't know for sure if Optiflex 7040 firmware had a bug, so I had to find a workaround that would mask the issue. I knew that I could remove multiple or duplicate entries in the NVRAM and BCD system store by using multiple bcdedit commands. The primary challenge was to determine the correct object GUIDs for multiple object entries that needed to be removed, so I turned to PowerShell for help

Based on the bcdedit /enum firmware output I knew that all duplicate firmware entries looked like the example above containing just three lines: identifier, description and badmemoryaccess.

All I had to do was to whip out a PowerShell script that would enumerate the firmware namespace objects in the system BCD store, identify duplicate GUID entries and execute bcdedit /delete command to delete the duplicate firmware GUIDs:

# Determine where to do the logging 
$tsenv = New-Object -COMObject Microsoft.SMS.TSEnvironment 
$logPath = $tsenv.Value("LogPath")  
$logFile = "$logPath\$($myInvocation.MyCommand).log"

# Start the logging 
Start-Transcript $logFile
Write-Host "Logging to $logFile"
 
# Start Main Code Here
$orphaned_guids = @()

Write-Host "$($myInvocation.MyCommand) - Enumerating the firmware namespace objects in the system BCD store."
Write-Host "$($myInvocation.MyCommand) - Executing bcdedit /enum firmware"

# Grab 3 lines preceding badmemoryaccess entry using a ForEach-Object loop
$bcd_store =  invoke-expression "bcdedit /enum firmware" | Select-String "badmemoryaccess" -Context 3,0

Write-Host "$($myInvocation.MyCommand) - Processing the list of GUID entries on the computer."

ForEach ($firmware in $bcd_store) {
    # Clean up entries using string replacement. 
    # The regular expressions '^identifier +' & '^description +' match a (sub)string starting with the word "identifier" / "description" followed by one or more spaces, which is replaced with the empty string
    $desc = $firmware.Context.Precontext[2] -replace '^description +'
    $guid = $firmware.Context.Precontext[1] -replace '^identifier +'
    $identifier = $firmware.Context.Precontext[0]

    Write-Host "$($myInvocation.MyCommand) - Found following entry."
    Write-Host "$($myInvocation.MyCommand) - Description: $desc"
    Write-Host "$($myInvocation.MyCommand) - GUID: $guid"

    # So now we only need to find invalid boot records.
    # Assume following: description contains "Windows Boot Manager" and the first line of the chunk -eq "----------------------------"
    # This will remove valid "Windows Boot Manager" entry from the array as well as other entries such as NICv4 and NICv6
    If ($identifier -eq "----------------------------" -and $desc -eq "Windows Boot Manager") {
	Write-Host "$($myInvocation.MyCommand) - Following orphaned GUID detected: $guid"
        $orphaned_guids += $guid
    }
}

Write-Host "$($myInvocation.MyCommand) - Prepairing to remove all duplicate entries from the BCD store."

#  remove all duplicate entries from the bcdstore
ForEach ($entry in $orphaned_guids) {
    Write-Host "$($myInvocation.MyCommand) - Following GUID will be removed $entry"
    $cmdline = 'bcdedit /delete $entry'
    Write-Host "$($myInvocation.MyCommand) - Executing $cmdline"
    invoke-expression $cmdline
}

Write-Host "$($myInvocation.MyCommand) - You can use use the bcdedit /enum firmware command to verify that all duplicate firmware entries have been removed."

# Stop logging 
Stop-Transcript

I added the script to my Windows 10 deployment task sequence and - to my immense satisfaction - duplicate firmware entries went away. The case was closed! As to why only one Dell model was affected by this issue, this part of the case remains a mystery to this day. As far as I was concerned, I had a workaround in place which ensured that the issue would not recur.

BootManagerCleaner.ps1

Read 7336 times Last modified on Monday, 25 September 2017 18:28
  1. Comments (0)

  2. Add yours
There are no comments posted here yet

Leave your comments

  1. Posting comment as a guest.
0 Characters
Attachments (0 / 3)
Share Your Location

Recent Posts

  • An alternative ESU MAK Activation Solution
    This blog post was shared with me by a colleague of mine, Daniel Dorner, a Microsoft Premier Field Engineer. It’s…
    Written on Wednesday, 04 December 2019 21:04
  • The Case of Missing UE-V Templates
    My customers often deal with unexpected Windows behavior and this case is no different. This particular one is especially interesting…
    Written on Tuesday, 03 September 2019 12:20
  • The Case of Corrupted Store Apps
    A few days ago I began experiencing issues with built-in Windows apps where various apps would flash open and close…
    Written on Wednesday, 14 August 2019 13:36
  • The Case of Changing Default Printer
    While I sometimes long for the day when I no longer have to deal with unexpected Windows 10 behavior, there’s…
    Written on Wednesday, 14 August 2019 20:36
  • Windows 10 1903: Useful Resources for IT Professionals
    Windows 10, version 1903 is now available via Windows Update for Business, Windows Server Update Services (WSUS) and the Volume…
    Written on Friday, 07 June 2019 11:21
  • Windows 10 1903 Built-In Apps: What to Keep
    The development of the Windows 10, version 1903 is finished and the update is now available for download from Visual…
    Written on Monday, 03 June 2019 06:59