Tag Archive for 'KiUserApcDispatcher'

[NT Inside Study] TLS , CreateThread vs _beginthread

처음 CRT Function이 만들어질 당시는 MultiThread 환경의 Application이 거의 존재하지 않았지요. 단일 Thread로 동작하는 Application이 거의 전부였습니다. 이러한 환경에서 개발된 CRT Function의 MultiThred 지원을 위해서 생각해낸 방법이 바로 TLS( Thread-Local Storage ) 를 사용하는 방법입니다.  현재의 CRT Function은 기존에 Glocal 또는 Static 으로 사용 하던 변수를 TLS를 이용하여 각 Thread 마다 할당함으로써 Thread-Safe 하도록 돕고 있습니다. 

CreateThread와 _beginthread의 가장 큰 차이점은 TLS를 사용하는 데 있습니다. CreateThread의 경우 실제로 Parameter의 인자를 바탕으로 Thread Execute합니다. 이런경우 만약 CRT Function에서 사용하고 있는 TSL의 Slot를 수동으로 Set하지 않으면 Exception 생길 수 있습니다.(이론상으론 .. ) 이와 바대로    _beginthread의 경우 CreateThread와 달리 CRT Function에서 사용하는 TSL의 Slot를 Set 해줌으로써 이를 해결해 주지요.

0:001> uf msvcrt!_beginthread
msvcrt!_beginthread:
77bea26e 8bff mov edi,edi
77bea270 55 push ebp
77bea271 8bec mov ebp,esp
77bea273 53 push ebx
77bea274 57 push edi
77bea275 8b7d08 mov edi,dword ptr [ebp+8]
77bea278 33db xor ebx,ebx
77bea27a 85ff test edi,edi
77bea27c 7510 jne msvcrt!_beginthread+0×20 (77bea28e)
msvcrt!_beginthread+0×10:
77bea27e e83950feff call msvcrt!_errno (77bcf2bc)
77bea283 c70016000000 mov dword ptr [eax],16h
77bea289 83c8ff or eax,0FFFFFFFFh
77bea28c eb70 jmp msvcrt!_beginthread+0×90 (77bea2fe)
msvcrt!_beginthread+0×20:
77bea28e 56 push esi
77bea28f 6888000000 push 88h
77bea294 6a01 push 1
77bea296 e8281effff call msvcrt!calloc (77bdc0c3)
77bea29b 8bf0 mov esi,eax
77bea29d 85f6 test esi,esi
77bea29f 59 pop ecx
77bea2a0 59 pop ecx
77bea2a1 7441 je msvcrt!_beginthread+0×76 (77bea2e4)
msvcrt!_beginthread+0×35:
77bea2a3 56 push esi
77bea2a4 e85ffcffff call msvcrt!_initptd (77be9f08)
77bea2a9 8b4510 mov eax,dword ptr [ebp+10h]
77bea2ac 59 pop ecx
77bea2ad 56 push esi
77bea2ae 6a04 push 4
77bea2b0 56 push esi
77bea2b1 68d7a1be77 push offset msvcrt!_endthread+0×43 (77bea1d7)
77bea2b6 ff750c push dword ptr [ebp+0Ch]
77bea2b9 897e4c mov dword ptr [esi+4Ch],edi
77bea2bc 6a00 push 0
77bea2be 894650 mov dword ptr [esi+50h],eax
77bea2c1 ff153412bc77 call dword ptr [msvcrt!_imp__CreateThread (77bc1234)]
77bea2c7 8bf8 mov edi,eax
77bea2c9 85ff test edi,edi
77bea2cb 897e04 mov dword ptr [esi+4],edi
77bea2ce 740c je msvcrt!_beginthread+0×6e (77bea2dc)
….

0:001> uf 77bea1d7
msvcrt!_endthread+0x43:
77bea1d7 6a0c push 0Ch
77bea1d9 68c840bc77 push offset msvcrt!`string'+0x24 (77bc40c8)
77bea1de e83dd2ffff call msvcrt!_SEH_prolog (77be7420)
77bea1e3 ff35ccfac077 push dword ptr [msvcrt!__tlsindex (77c0facc)]
77bea1e9 ff154410bc77 call dword ptr [msvcrt!_imp__TlsGetValue (77bc1044)]
77bea1ef 8bf0 mov esi,eax
77bea1f1 85f6 test esi,esi
77bea1f3 751d jne msvcrt!_endthread+0×7e (77bea212)
msvcrt!_endthread+0x61:
77bea1f5 8b7508 mov esi,dword ptr [ebp+8]
77bea1f8 56 push esi
77bea1f9 ff35ccfac077 push dword ptr [msvcrt!__tlsindex (77c0facc)]
77bea1ff ff154010bc77 call dword ptr [msvcrt!_imp__TlsSetValue (77bc1040)]
77bea205 85c0 test eax,eax
77bea207 7525 jne msvcrt!_endthread+0×9a (77bea22e)

위의 Asm Code를 보면 알 수 있듯이  _beginthread는 실제 수행할 Thread Entry의 Wrapper Function을 만들어 내부에서 TLS Slot를 확인하여 TLS Slot Validation를 보장해줍니다.

그러나 !!! 우리가 Visual C++를 통해서 실제 개발을 할때는 주로 CreateThread를 호출하고 실제로 Exception이 발생하지 않습니다.  그럼 왜 그럴까 ?? 
 Windows는 Thread를 생성 초기화 할 때는 Kernel To User Callback( ntdll!KiUserApcDispatcher )을 통해서 UserThread가 사용하고자 하는 Resource를 초기화 합니다.

00 011ffb9c 10204beb 00000002 7c809740 7c809740 kernel32!TlsSetValue (FPO: [Non-Fpo])
01 011ffbb0 102032a6 27316ed0 00000000 011ffc24 MSVCR80D!__set_flsgetvalue+0×3b (FPO: [Non-Fpo])
02 011ffbfc 10202faf 10200000 00000002 00000000 MSVCR80D!_CRTDLL_INIT+0×316 (FPO: [Non-Fpo])
03 011ffc10 7c9311a7 10200000 00000002 00000000 MSVCR80D!_CRTDLL_INIT+0×1f (FPO: [Non-Fpo])
04 011ffc30 7c948f65 10202f90 10200000 00000002 ntdll!LdrpCallInitRoutine+0×14
05 011ffca4 7c948dde 011ffd30 011ffd30 010003f0 ntdll!LdrpInitializeThread+0xc0 (FPO: [Non-Fpo])

06 011ffd1c 7c93eac7 011ffd30 7c930000 00000000 ntdll!_LdrpInitialize+0×219 (FPO: [Non-Fpo])
07 00000000 00000000 00000000 00000000 00000000 ntdll!KiUserApcDispatcher+0×7
ntdll!LdrpInitializeThread:
7c948e9a 6a40 push 40h
7c948e9c 68d08e947c push offset ntdll!`string’+0×88 (7c948ed0)
7c948ea1 e81c5fffff call ntdll!_SEH_prolog (7c93edc2)
7c948ea6 64a118000000 mov eax,dword ptr fs:[00000018h] // TEB
7c948eac 8b5830 mov ebx,dword ptr [eax+30h] // PEB
7c948eaf 895ddc mov dword ptr [ebp-24h],ebx
7c948eb2 803d30c09a7c00 cmp byte ptr [ntdll!LdrpShutdownInProgress (7c9ac030)],0
7c948eb9 0f85c3000000 jne ntdll!LdrpInitializeThread+0×136 (7c948f82)
ntdll!LdrpInitializeThread+0x25:
7c948ebf e859ffffff call ntdll!LdrpAllocateTls (7c948e1d)
7c948ec4 8b430c mov eax,dword ptr [ebx+0Ch] // PEB_LDR_DATA
7c948ec7 8b7014 mov esi,dword ptr [eax+14h] //InMemoryOrderModuleList
7c948eca eb1c jmp ntdll!LdrpInitializeThread+0×30 (7c948ee8)
... 생략
ntdll!LdrpInitializeThread+0x56:
7c948f0a 8b4e14 mov ecx,dword ptr [esi+14h]
7c948f0d 894de0 mov dword ptr [ebp-20h],ecx //DLL Entry Point
7c948f10 85c9 test ecx,ecx
7c948f12 745a je ntdll!LdrpInitializeThread+0xc9 (7c948f6e)
… 생략
ntdll!LdrpInitializeThread+0xa9:
7c948f4e 803d30c09a7c00 cmp byte ptr [ntdll!LdrpShutdownInProgress (7c9ac030)],0
7c948f55 750e jne ntdll!LdrpInitializeThread+0xc0 (7c948f65)
ntdll!LdrpInitializeThread+0xb2:
7c948f57 57 push edi
7c948f58 6a02 push 2
7c948f5a ff7610 push dword ptr [esi+10h]
7c948f5d ff75e0 push dword ptr [ebp-20h]
7c948f60 e82e82feff call ntdll!LdrpCallInitRoutine (7c931193)
ntdll!LdrpInitializeThread+0xc0:
7c948f65 834dfcff or dword ptr [ebp-4],0FFFFFFFFh
7c948f69 e81dffffff call ntdll!LdrpInitializeThread+0xd6 (7c948e8b)
... 생략

ntdll!LdrpCallInitRoutine:
7c931193 55 push ebp
7c931194 8bec mov ebp,esp
7c931196 56 push esi
7c931197 57 push edi
7c931198 53 push ebx
7c931199 8bf4 mov esi,esp
7c93119b ff7514 push dword ptr [ebp+14h]
7c93119e ff7510 push dword ptr [ebp+10h]
7c9311a1 ff750c push dword ptr [ebp+0Ch]
7c9311a4 ff5508 call dword ptr [ebp+8]
7c9311a7 8be6 mov esp,esi
7c9311a9 5b pop ebx
7c9311aa 5f pop edi
7c9311ab 5e pop esi
7c9311ac 5d pop ebp
7c9311ad c21000 ret 10h

ntdll!LdrpInitializeThread == > ntdll!LdrpCallInitRoutine 를 통해서 InMemoryOrderModuleList 참조하고 각 Dll의 initialize Code를 수행하는것을 확인 할 수 있습니다.  그리고 각 구조체들의 실제 구조를 살펴 보면 아래와 같이 참조 한다는 것을 알 수 있습니다.

lkd> dt _TEB
nt!_TEB
+0x000 NtTib : _NT_TIB
+0x01c EnvironmentPointer : Ptr32 Void
+0x020 ClientId : _CLIENT_ID
+0x028 ActiveRpcHandle : Ptr32 Void
+0x02c ThreadLocalStoragePointer : Ptr32 Void
+0×030 ProcessEnvironmentBlock : Ptr32 _PEB
+0×034 LastErrorValue : Uint4B
+0×038 CountOfOwnedCriticalSections : Uint4B
+0×03c CsrClientThread : Ptr32 Void
+0×040 Win32ThreadInfo : Ptr32 Void
+0×044 User32Reserved : [26] Uint4B
+0×0ac UserReserved : [5] Uint4B
+0×0c0 WOW32Reserved : Ptr32 Void
+0×0c4 CurrentLocale : Uint4B
+0×0c8 FpSoftwareStatusRegister : Uint4B
+0×0cc SystemReserved1 : [54] Ptr32 Void
+0×1a4 ExceptionCode : Int4B
+0×1a8 ActivationContextStack : _ACTIVATION_CONTEXT_STACK
+0×1bc SpareBytes1 : [24] UChar
+0×1d4 GdiTebBatch : _GDI_TEB_BATCH
+0×6b4 RealClientId : _CLIENT_ID
+0×6bc GdiCachedProcessHandle : Ptr32 Void
+0×6c0 GdiClientPID : Uint4B
+0×6c4 GdiClientTID : Uint4B
+0×6c8 GdiThreadLocalInfo : Ptr32 Void
+0×6cc Win32ClientInfo : [62] Uint4B
+0×7c4 glDispatchTable : [233] Ptr32 Void
+0xb68 glReserved1 : [29] Uint4B
+0xbdc glReserved2 : Ptr32 Void
+0xbe0 glSectionInfo : Ptr32 Void
+0xbe4 glSection : Ptr32 Void
+0xbe8 glTable : Ptr32 Void
+0xbec glCurrentRC : Ptr32 Void
+0xbf0 glContext : Ptr32 Void
+0xbf4 LastStatusValue : Uint4B
+0xbf8 StaticUnicodeString : _UNICODE_STRING
+0xc00 StaticUnicodeBuffer : [261] Uint2B
+0xe0c DeallocationStack : Ptr32 Void
+0xe10 TlsSlots : [64] Ptr32 Void
+0xf10 TlsLinks : _LIST_ENTRY
+0xf18 Vdm : Ptr32 Void
+0xf1c ReservedForNtRpc : Ptr32 Void
+0xf20 DbgSsReserved : [2] Ptr32 Void
+0xf28 HardErrorsAreDisabled : Uint4B
+0xf2c Instrumentation : [16] Ptr32 Void
+0xf6c WinSockData : Ptr32 Void
+0xf70 GdiBatchCount : Uint4B
+0xf74 InDbgPrint : UChar
+0xf75 FreeStackOnTermination : UChar
+0xf76 HasFiberData : UChar
+0xf77 IdealProcessor : UChar
+0xf78 Spare3 : Uint4B
+0xf7c ReservedForPerf : Ptr32 Void
+0xf80 ReservedForOle : Ptr32 Void
+0xf84 WaitingOnLoaderLock : Uint4B
+0xf88 Wx86Thread : _Wx86ThreadState
+0xf94 TlsExpansionSlots : Ptr32 Ptr32 Void
+0xf98 ImpersonationLocale : Uint4B
+0xf9c IsImpersonating : Uint4B
+0xfa0 NlsCache : Ptr32 Void
+0xfa4 pShimData : Ptr32 Void
+0xfa8 HeapVirtualAffinity : Uint4B
+0xfac CurrentTransactionHandle : Ptr32 Void
+0xfb0 ActiveFrame : Ptr32 _TEB_ACTIVE_FRAME
+0xfb4 SafeThunkCall : UChar
+0xfb5 BooleanSpare : [3] UChar


lkd> dt _PEB
nt!_PEB
+0x000 InheritedAddressSpace : UChar
+0x001 ReadImageFileExecOptions : UChar
+0x002 BeingDebugged : UChar
+0x003 SpareBool : UChar
+0x004 Mutant : Ptr32 Void
+0x008 ImageBaseAddress : Ptr32 Void
+0×00c Ldr : Ptr32 _PEB_LDR_DATA
+0×010 ProcessParameters : Ptr32 _RTL_USER_PROCESS_PARAMETERS
+0×014 SubSystemData : Ptr32 Void
+0×018 ProcessHeap : Ptr32 Void
+0×01c FastPebLock : Ptr32 _RTL_CRITICAL_SECTION
+0×020 FastPebLockRoutine : Ptr32 Void
+0×024 FastPebUnlockRoutine : Ptr32 Void
+0×028 EnvironmentUpdateCount : Uint4B
+0×02c KernelCallbackTable : Ptr32 Void
+0×030 SystemReserved : [1] Uint4B
+0×034 AtlThunkSListPtr32 : Uint4B
+0×038 FreeList : Ptr32 _PEB_FREE_BLOCK
+0×03c TlsExpansionCounter : Uint4B
+0×040 TlsBitmap : Ptr32 Void
+0×044 TlsBitmapBits : [2] Uint4B
+0×04c ReadOnlySharedMemoryBase : Ptr32 Void
+0×050 ReadOnlySharedMemoryHeap : Ptr32 Void
+0×054 ReadOnlyStaticServerData : Ptr32 Ptr32 Void
+0×058 AnsiCodePageData : Ptr32 Void
+0×05c OemCodePageData : Ptr32 Void
+0×060 UnicodeCaseTableData : Ptr32 Void
+0×064 NumberOfProcessors : Uint4B
+0×068 NtGlobalFlag : Uint4B
+0×070 CriticalSectionTimeout : _LARGE_INTEGER
+0×078 HeapSegmentReserve : Uint4B
+0×07c HeapSegmentCommit : Uint4B
+0×080 HeapDeCommitTotalFreeThreshold : Uint4B
+0×084 HeapDeCommitFreeBlockThreshold : Uint4B
+0×088 NumberOfHeaps : Uint4B
+0×08c MaximumNumberOfHeaps : Uint4B
+0×090 ProcessHeaps : Ptr32 Ptr32 Void
+0×094 GdiSharedHandleTable : Ptr32 Void
+0×098 ProcessStarterHelper : Ptr32 Void
+0×09c GdiDCAttributeList : Uint4B
+0×0a0 LoaderLock : Ptr32 Void
+0×0a4 OSMajorVersion : Uint4B
+0×0a8 OSMinorVersion : Uint4B
+0×0ac OSBuildNumber : Uint2B
+0×0ae OSCSDVersion : Uint2B
+0×0b0 OSPlatformId : Uint4B
+0×0b4 ImageSubsystem : Uint4B
+0×0b8 ImageSubsystemMajorVersion : Uint4B
+0×0bc ImageSubsystemMinorVersion : Uint4B
+0×0c0 ImageProcessAffinityMask : Uint4B
+0×0c4 GdiHandleBuffer : [34] Uint4B
+0×14c PostProcessInitRoutine : Ptr32 void
+0×150 TlsExpansionBitmap : Ptr32 Void
+0×154 TlsExpansionBitmapBits : [32] Uint4B
+0×1d4 SessionId : Uint4B
+0×1d8 AppCompatFlags : _ULARGE_INTEGER
+0×1e0 AppCompatFlagsUser : _ULARGE_INTEGER
+0×1e8 pShimData : Ptr32 Void
+0×1ec AppCompatInfo : Ptr32 Void
+0×1f0 CSDVersion : _UNICODE_STRING
+0×1f8 ActivationContextData : Ptr32 Void
+0×1fc ProcessAssemblyStorageMap : Ptr32 Void
+0×200 SystemDefaultActivationContextData : Ptr32 Void
+0×204 SystemAssemblyStorageMap : Ptr32 Void
+0×208 MinimumStackCommit : Uint4B


lkd> dt _PEB_LDR_DATA
nt!_PEB_LDR_DATA
+0x000 Length : Uint4B
+0x004 Initialized : UChar
+0x008 SsHandle : Ptr32 Void
+0x00c InLoadOrderModuleList : _LIST_ENTRY
+0×014 InMemoryOrderModuleList : _LIST_ENTRY
+0×01c InInitializationOrderModuleList : _LIST_ENTRY
+0×024 EntryInProgress : Ptr32 Void

lkd> dt _LDR_DATA_TABLE_ENTRY
nt!_LDR_DATA_TABLE_ENTRY
+0x000 InLoadOrderLinks : _LIST_ENTRY
+0×008 InMemoryOrderLinks : _LIST_ENTRY
+0×010 InInitializationOrderLinks : _LIST_ENTRY
+0×018 DllBase : Ptr32 Void
+0×01c EntryPoint : Ptr32 Void
+0×020 SizeOfImage : Uint4B
+0×024 FullDllName : _UNICODE_STRING
+0×02c BaseDllName : _UNICODE_STRING
+0×034 Flags : Uint4B
+0×038 LoadCount : Uint2B
+0×03a TlsIndex : Uint2B
+0×03c HashLinks : _LIST_ENTRY
+0×03c SectionPointer : Ptr32 Void
+0×040 CheckSum : Uint4B
+0×044 TimeDateStamp : Uint4B
+0×044 LoadedImports : Ptr32 Void
+0×048 EntryPointActivationContext : Ptr32 Void
+0×04c PatchInformation : Ptr32 Void

위와 같은 결과로 볼때 CRT Function를 Thread안에서 사용하고자 한다면 _beginthread를 권장하지만 구지 사용하지 않고 CreateThread를 사용하더라도 큰 문제는 없을 것이라 생각됩니다.

[NT Inside Study] Kernel To User Callback Function의 Address는 어디에 가지고 있나 ??

아래의 3가지 Function은 Kernel To User Callback을 대표하는 함수 입니다. 그러면 이 녀석들의 Address는 Kernel의 어디에서 들고 있을까 ?

7c93ead0 ntdll!KiUserCallbackDispatcher = <no type information>
7c93eac0 ntdll!KiUserApcDispatcher = <no type information>
7c93eaec ntdll!KiUserExceptionDispatcher = <no type information>

간단히 nt Symbol에서 확인이 가능합니다.

lkd> X nt!KeUser*
80563474 nt!KeUserCallbackDispatcher = <no type information>
80563478 nt!KeUserApcDispatcher = <no type information>
805701cc nt!KeUserModeCallback = <no type information>
80563470 nt!KeUserExceptionDispatcher = <no type information>

 커널쪽 심볼중 이름이 비슷한 세녀석이 있지요. 실제로 이 세녀석의 Memory를 열어 보면 User Mode Callback의 Address가 담겨 있는것을 확인 할 수가 있습니다.

lkd> dd 80563470
80563470 7c93eaec 7c93ead0 7c93eac0 00002626
80563480 00000000 00000000 00000000 00000000
80563490 00000000 00000000 00000000 00000000
805634a0 00000000 00000000 00000000 00000000
805634b0 00000000 00000000 00000000 00000000
805634c0 804e68b0 00000000 0000011c 80519fc4
805634d0 bf999980 00000000 0000029b bf99a690
805634e0 00000000 00000000 00000000 00000000

User Callback Function 세녀석의 주소가 나란히 저장되어 있는것을 확인할 수 가 있습니다.