My first step was to run bcdedit /enum firmware command to enumerate the firmware namespace objects in BCD. The output showed entries for local devices, such as a CD-ROM and NIC as well as one valid "Windows Boot Manager" entry. In addition, a large number of orphaned "Windows Boot Manager" entries was present in the system boot configuration data (BCD).
The following example is snippet of the output from one of the affected systems:
Firmware Boot Manager
---------------------
identifier {fwbootmgr}
displayorder {bootmgr}
{83222746-075c-11e6-b382-64006a688434}
{83222757-075c-11e6-b382-64006a688434}
{83222755-075c-11e6-b382-64006a688434}
{83222754-075c-11e6-b382-64006a688434}
{83222753-075c-11e6-b382-64006a688434}
{83222752-075c-11e6-b382-64006a688434}
{83222751-075c-11e6-b382-64006a688434}
{83222750-075c-11e6-b382-64006a688434}
{8322274f-075c-11e6-b382-64006a688434}
{8322274e-075c-11e6-b382-64006a688434}
{8322274d-075c-11e6-b382-64006a688434}
{8322274c-075c-11e6-b382-64006a688434}
{8322274b-075c-11e6-b382-64006a688434}
{8322274a-075c-11e6-b382-64006a688434}
{83222741-075c-11e6-b382-64006a688434}
{83222742-075c-11e6-b382-64006a688434}
{83222743-075c-11e6-b382-64006a688434}
{83222744-075c-11e6-b382-64006a688434}
{83222745-075c-11e6-b382-64006a688434}
{83222747-075c-11e6-b382-64006a688434}
{83222748-075c-11e6-b382-64006a688434}
{83222749-075c-11e6-b382-64006a688434}
timeout 2
Windows-Start-Manager
---------------------
identifier {bootmgr}
device partition=\Device\HarddiskVolume1
path \EFI\Microsoft\Boot\bootmgfw.efi
description Windows Boot Manager
locale de-DE
inherit {globalsettings}
badmemoryaccess Yes
default {current}
resumeobject {8322273f-075c-11e6-b382-64006a688434}
displayorder {current}
toolsdisplayorder {memdiag}
timeout 30
Firmware Application (101fffff)
----------------------------
identifier {83222741-075c-11e6-b382-64006a688434}
description Windows Boot Manager
badmemoryaccess Yes
Firmware Application (101fffff)
----------------------------
identifier {83222742-075c-11e6-b382-64006a688434}
description Windows Boot Manager
badmemoryaccess Yes
Firmware Application (101fffff)
----------------------------
identifier {83222748-075c-11e6-b382-64006a688434}
description Onboard NIC(IPV4)
badmemoryaccess Yes
This particular system was used to test Windows 10 image, so it was being reinstalled fairly often. I suspected that a duplicate "Windows Boot Manager" entry was created during OSD. Multiple installations resulted in many duplicate entries. I likely could have worked around the problem for the time being by setting boot menu timeout to 0 resulting in the default operating system starting instantly, but knew that if I didn’t determine the root cause and address it, the customer will run into the issue at some point. In any case, stooping to that kind of troubleshooting hack is beneath my dignity. I therefore set out to investigate
From past experience I knew that on some Unified Extensible Firmware Interface (UEFI)-based systems, the firmware creates non-volatile random access memory (NVRAM) entries for local devices when the computer boots. When MDT tries to create boot loader entries it executes BCDBoot, followed by bcdedit:
\\MDT01\DeploymentShare$\Tools\X86\BCDBoot.exe C:\Windows /l de-DE /s V: X:\WINDOWS\SYSTEM32\bcdedit.exe /Store "V:\boot\bcd" /timeout 0
When bcdedit opens the BCD, it compares entries in NVRAM with entries in BCD. Entries in NVRAM that were created by the firmware that do not exist in BCD are added to the system BCD. When bcdedit closes the system BCD, any boot manager entries in BCD that are not in NVRAM are added to NVRAM. This behavior might result in multiple firmware entries for "Windows Boot Manager" with different GUIDs on some devices that are similar to the following example:
Firmware Application (101fffff)
----------------------------
identifier {83222741-075c-11e6-b382-64006a688434}
description Windows Boot Manager
badmemoryaccess Yes
Firmware Application (101fffff)
----------------------------
identifier {83222742-075c-11e6-b382-64006a688434}
description Windows Boot Manager
badmemoryaccess Yes
The first thing I do when I suspect that some kind of model specific problem is to contact the product group. Unfortunately, none of my contacts were aware of the issue. Adding to the mystery only one particular Dell model was affected - even the sister model, Dell Optiplex 7040 Micro, was not showing the systems. I was at a dead end. I couldn't know for sure if Optiflex 7040 firmware had a bug, so I had to find a workaround that would mask the issue. I knew that I could remove multiple or duplicate entries in the NVRAM and BCD system store by using multiple bcdedit commands. The primary challenge was to determine the correct object GUIDs for multiple object entries that needed to be removed, so I turned to PowerShell for help
Based on the bcdedit /enum firmware output I knew that all duplicate firmware entries looked like the example above containing just three lines: identifier, description and badmemoryaccess.
All I had to do was to whip out a PowerShell script that would enumerate the firmware namespace objects in the system BCD store, identify duplicate GUID entries and execute bcdedit /delete command to delete the duplicate firmware GUIDs:
# Determine where to do the logging
$tsenv = New-Object -COMObject Microsoft.SMS.TSEnvironment
$logPath = $tsenv.Value("LogPath")
$logFile = "$logPath\$($myInvocation.MyCommand).log"
# Start the logging
Start-Transcript $logFile
Write-Host "Logging to $logFile"
# Start Main Code Here
$orphaned_guids = @()
Write-Host "$($myInvocation.MyCommand) - Enumerating the firmware namespace objects in the system BCD store."
Write-Host "$($myInvocation.MyCommand) - Executing bcdedit /enum firmware"
# Grab 3 lines preceding badmemoryaccess entry using a ForEach-Object loop
$bcd_store = invoke-expression "bcdedit /enum firmware" | Select-String "badmemoryaccess" -Context 3,0
Write-Host "$($myInvocation.MyCommand) - Processing the list of GUID entries on the computer."
ForEach ($firmware in $bcd_store) {
# Clean up entries using string replacement.
# The regular expressions '^identifier +' & '^description +' match a (sub)string starting with the word "identifier" / "description" followed by one or more spaces, which is replaced with the empty string
$desc = $firmware.Context.Precontext[2] -replace '^description +'
$guid = $firmware.Context.Precontext[1] -replace '^identifier +'
$identifier = $firmware.Context.Precontext[0]
Write-Host "$($myInvocation.MyCommand) - Found following entry."
Write-Host "$($myInvocation.MyCommand) - Description: $desc"
Write-Host "$($myInvocation.MyCommand) - GUID: $guid"
# So now we only need to find invalid boot records.
# Assume following: description contains "Windows Boot Manager" and the first line of the chunk -eq "----------------------------"
# This will remove valid "Windows Boot Manager" entry from the array as well as other entries such as NICv4 and NICv6
If ($identifier -eq "----------------------------" -and $desc -eq "Windows Boot Manager") {
Write-Host "$($myInvocation.MyCommand) - Following orphaned GUID detected: $guid"
$orphaned_guids += $guid
}
}
Write-Host "$($myInvocation.MyCommand) - Prepairing to remove all duplicate entries from the BCD store."
# remove all duplicate entries from the bcdstore
ForEach ($entry in $orphaned_guids) {
Write-Host "$($myInvocation.MyCommand) - Following GUID will be removed $entry"
$cmdline = 'bcdedit /delete $entry'
Write-Host "$($myInvocation.MyCommand) - Executing $cmdline"
invoke-expression $cmdline
}
Write-Host "$($myInvocation.MyCommand) - You can use use the bcdedit /enum firmware command to verify that all duplicate firmware entries have been removed."
# Stop logging
Stop-Transcript
I added the script to my Windows 10 deployment task sequence and - to my immense satisfaction - duplicate firmware entries went away. The case was closed! As to why only one Dell model was affected by this issue, this part of the case remains a mystery to this day. As far as I was concerned, I had a workaround in place which ensured that the issue would not recur.
