UE4对象系统_序列化和uasset文件格式
2017-08-24 本文已影响0人
蛋求疼
虚幻的序列化这块是个人比较喜欢的技术点，个人在工作中也山寨了个简化版，UE4延续了UE3的序列化方案。它使用了访问者模式(Vistor Pattern)，将序列化的存档接口抽象化，其中FArchive为访问者, 其它实现了void Serialize( FArchive& Ar )接口的类为被访问者。FArchive可以是磁盘文件访问, 内存统计，对象统计等功能。
FArchive
FArchive的类继承体系如下：
Paste_Image.png
定义接口如下:
/**
* Base class for archives that can be used for loading, saving, and garbage
* collecting in a byte order neutral way.
*/
class CORE_API FArchive
{
public:
/** Default constructor. */
FArchive();
/** Copy constructor. */
FArchive(const FArchive&);
/**
* Copy assignment operator.
*
* @param ArchiveToCopy The archive to copy from.
*/
FArchive& operator=(const FArchive& ArchiveToCopy);
/** Destructor. */
virtual ~FArchive();
public:
virtual FArchive& operator<<(class FName& Value);
virtual FArchive& operator<<(class FText& Value);
virtual FArchive& operator<<(class UObject*& Value);
virtual FArchive& operator<<(class FLazyObjectPtr& Value);
virtual FArchive& operator<<(class FAssetPtr& Value);
virtual FArchive& operator<<(struct FStringAssetReference& Value);
virtual FArchive& operator<<(struct FWeakObjectPtr& Value);
virtual void ForceBlueprintFinalization() {}
public:
FORCEINLINE friend FArchive& operator<<(FArchive& Ar, ANSICHAR& Value);
FORCEINLINE friend FArchive& operator<<(FArchive& Ar, WIDECHAR& Value);
FORCEINLINE friend FArchive& operator<<(FArchive& Ar, uint8& Value);
template<class TEnum>
FORCEINLINE friend FArchive& operator<<(FArchive& Ar, TEnumAsByte<TEnum>& Value);
FORCEINLINE friend FArchive& operator<<(FArchive& Ar, int8& Value);
FORCEINLINE friend FArchive& operator<<(FArchive& Ar, uint16& Value);
FORCEINLINE friend FArchive& operator<<(FArchive& Ar, int16& Value);
FORCEINLINE friend FArchive& operator<<(FArchive& Ar, uint32& Value);
FORCEINLINE friend FArchive& operator<<(FArchive& Ar, bool& D);
FORCEINLINE friend FArchive& operator<<(FArchive& Ar, int32& Value);
FORCEINLINE friend FArchive& operator<<(FArchive& Ar, long& Value);
FORCEINLINE friend FArchive& operator<<( FArchive& Ar, float& Value);
FORCEINLINE friend FArchive& operator<<(FArchive& Ar, double& Value);
FORCEINLINE friend FArchive& operator<<(FArchive &Ar, uint64& Value);
/*FORCEINLINE*/friend FArchive& operator<<(FArchive& Ar, int64& Value);
template <
typename EnumType,
typename = typename TEnableIf<TIsEnumClass<EnumType>::Value>::Type
>
FORCEINLINE friend FArchive& operator<<(FArchive& Ar, EnumType& Value)
{
return Ar << (__underlying_type(EnumType)&)Value;
}
friend FArchive& operator<<(FArchive& Ar, struct FIntRect& Value);
friend CORE_API FArchive& operator<<(FArchive& Ar, FString& Value);
public:
virtual void Serialize(void* V, int64 Length) ;
virtual void SerializeBits(void* V, int64 LengthBits);
virtual void SerializeInt(uint32& Value, uint32 Max)
/** Packs int value into bytes of 7 bits with 8th bit for 'more' */
virtual void SerializeIntPacked(uint32& Value);
virtual void Preload(UObject* Object) { }
virtual void CountBytes(SIZE_T InNum, SIZE_T InMax) { }
virtual FString GetArchiveName() const;
virtual class FLinker* GetLinker()
{
return nullptr;
}
virtual int64 Tell()；
virtual int64 TotalSize()；
virtual bool AtEnd()；
virtual void Seek(int64 InPos) { }
virtual void AttachBulkData(UObject* Owner, FUntypedBulkData* BulkData) { }
virtual void DetachBulkData(FUntypedBulkData* BulkData, bool bEnsureBulkDataIsLoaded) { }
/**
* Sets mapping from offsets/ sizes that are going to be used for seeking and serialization to what
* is actually stored on disk. If the archive supports dealing with compression in this way it is
* going to return true.
*
* @param CompressedChunks Pointer to array containing information about [un]compressed chunks
* @param CompressionFlags Flags determining compression format associated with mapping
*
* @return true if archive supports translating offsets & uncompressing on read, false otherwise
*/
virtual bool SetCompressionMap(TArray<struct FCompressedChunk>* CompressedChunks, ECompressionFlags CompressionFlags)；
virtual void Flush() { }
virtual bool Close()；
virtual bool GetError()；
void SetError() ；
/**
* Serializes and compresses/ uncompresses data. This is a shared helper function for compression
* support. The data is saved in a way compatible with FIOSystem::LoadCompressedData.
*
* @param V Data pointer to serialize data from/ to
* @param Length Length of source data if we're saving, unused otherwise
* @param Flags Flags to control what method to use for [de]compression and optionally control memory vs speed when compressing
* @param bTreatBufferAsFileReader true if V is actually an FArchive, which is used when saving to read data - helps to avoid single huge allocations of source data
* @param bUsePlatformBitWindow use a platform specific bitwindow setting
*/
void SerializeCompressed(void* V, int64 Length, ECompressionFlags Flags, bool bTreatBufferAsFileReader = false, bool bUsePlatformBitWindow = false);
FORCEINLINE bool IsByteSwapping(); // 平台的大端、小端
// Used to do byte swapping on small items. This does not happen usually, so we don't want it inline
void ByteSwap(void* V, int32 Length);
FORCEINLINE FArchive& ByteOrderSerialize(void* V, int32 Length);
/** Sets a flag indicating that this archive contains code. */
void ThisContainsCode();
/** Sets a flag indicating that this archive contains a ULevel or UWorld object. */
void ThisContainsMap() ;
/** Sets a flag indicating that this archive contains data required to be gathered for localization. */
void ThisRequiresLocalizationGather();
/** Sets a flag indicating that this archive is currently serializing class/struct defaults. */
void StartSerializingDefaults() ;
/** Indicate that this archive is no longer serializing class/struct defaults. */
void StopSerializingDefaults() ;
/**
* Called when an object begins serializing property data using script serialization.
*/
virtual void MarkScriptSerializationStart(const UObject* Obj) { }
/**
* Called when an object stops serializing property data using script serialization.
*/
virtual void MarkScriptSerializationEnd(const UObject* Obj) { }
FORCEINLINE bool IsLoading() const
{
return ArIsLoading;
}
FORCEINLINE bool IsSaving() const
{
return ArIsSaving;
}
FORCEINLINE bool IsTransacting() const
{
if (FPlatformProperties::HasEditorOnlyData())
{
return ArIsTransacting;
}
else
{
return false;
}
}
FORCEINLINE bool WantBinaryPropertySerialization() const
{
return ArWantBinaryPropertySerialization;
}
FORCEINLINE bool IsForcingUnicode() const
{
return ArForceUnicode;
}
FORCEINLINE bool IsPersistent() const
{
return ArIsPersistent;
}
FORCEINLINE bool IsError() const
{
return ArIsError;
}
FORCEINLINE bool IsCriticalError() const
{
return ArIsCriticalError;
}
FORCEINLINE bool ContainsCode() const
{
return ArContainsCode;
}
FORCEINLINE bool ContainsMap() const
{
return ArContainsMap;
}
FORCEINLINE bool RequiresLocalizationGather() const
{
return ArRequiresLocalizationGather;
}
FORCEINLINE bool ForceByteSwapping() const
{
return ArForceByteSwapping;
}
FORCEINLINE bool IsSerializingDefaults() const
{
return (ArSerializingDefaults > 0) ? true : false;
}
FORCEINLINE bool IsIgnoringArchetypeRef() const
{
return ArIgnoreArchetypeRef;
}
FORCEINLINE bool DoDelta() const
{
return !ArNoDelta;
}
FORCEINLINE bool IsIgnoringOuterRef() const
{
return ArIgnoreOuterRef;
}
FORCEINLINE bool IsIgnoringClassGeneratedByRef() const
{
return ArIgnoreClassGeneratedByRef;
}
FORCEINLINE bool IsIgnoringClassRef() const
{
return ArIgnoreClassRef;
}
FORCEINLINE bool IsAllowingLazyLoading() const
{
return ArAllowLazyLoading;
}
FORCEINLINE bool IsObjectReferenceCollector() const
{
return ArIsObjectReferenceCollector;
}
FORCEINLINE bool IsModifyingWeakAndStrongReferences() const
{
return ArIsModifyingWeakAndStrongReferences;
}
FORCEINLINE bool IsCountingMemory() const
{
return ArIsCountingMemory;
}
FORCEINLINE uint32 GetPortFlags() const
{
return ArPortFlags;
}
private:
/** Copies all of the members except CustomVersionContainer */
void CopyTrivialFArchiveStatusMembers(const FArchive& ArchiveStatusToCopy);
public:
/** Whether this archive is for loading data. */
uint8 ArIsLoading : 1;
/** Whether this archive is for saving data. */
uint8 ArIsSaving : 1;
/** Whether archive is transacting. */
uint8 ArIsTransacting : 1;
/** Whether this archive wants properties to be serialized in binary form instead of tagged. */
uint8 ArWantBinaryPropertySerialization : 1;
/** Whether this archive wants to always save strings in unicode format */
uint8 ArForceUnicode : 1;
/** Whether this archive saves to persistent storage. */
uint8 ArIsPersistent : 1;
/** Whether this archive contains errors. */
uint8 ArIsError : 1;
/** Whether this archive contains critical errors. */
uint8 ArIsCriticalError : 1;
/** Quickly tell if an archive contains script code. */
uint8 ArContainsCode : 1;
/** Used to determine whether FArchive contains a level or world. */
uint8 ArContainsMap : 1;
/** Used to determine whether FArchive contains data required to be gathered for localization. */
uint8 ArRequiresLocalizationGather : 1;
/** Whether we should forcefully swap bytes. */
uint8 ArForceByteSwapping : 1;
/** If true, we will not serialize the ObjectArchetype reference in UObject. */
uint8 ArIgnoreArchetypeRef : 1;
/** If true, we will not serialize the ObjectArchetype reference in UObject. */
uint8 ArNoDelta : 1;
/** If true, we will not serialize the Outer reference in UObject. */
uint8 ArIgnoreOuterRef : 1;
/** If true, we will not serialize ClassGeneratedBy reference in UClass. */
uint8 ArIgnoreClassGeneratedByRef : 1;
/** If true, UObject::Serialize will skip serialization of the Class property. */
uint8 ArIgnoreClassRef : 1;
/** Whether to allow lazy loading. */
uint8 ArAllowLazyLoading : 1;
/** Whether this archive only cares about serializing object references. */
uint8 ArIsObjectReferenceCollector : 1;
/** Whether a reference collector is modifying the references and wants both weak and strong ones */
uint8 ArIsModifyingWeakAndStrongReferences : 1;
/** Whether this archive is counting memory and therefore wants e.g. TMaps to be serialized. */
uint8 ArIsCountingMemory : 1;
/** Whether bulk data serialization should be skipped or not. */
uint8 ArShouldSkipBulkData : 1;
/** Whether editor only properties are being filtered from the archive (or has been filtered). */
uint8 ArIsFilterEditorOnly : 1;
/** Whether this archive is saving/loading game state */
uint8 ArIsSaveGame : 1;
/** Set TRUE to use the custom property list attribute for serialization. */
uint8 ArUseCustomPropertyList : 1;
/** Whether we are currently serializing defaults. > 0 means yes, <= 0 means no. */
int32 ArSerializingDefaults;
/** Modifier flags that be used when serializing UProperties */
uint32 ArPortFlags;
/** Max size of data that this archive is allowed to serialize. */
int64 ArMaxSerializeSize;
protected:
/** Holds the archive version. */
int32 ArUE4Ver;
/** Holds the archive version for licensees. */
int32 ArLicenseeUE4Ver;
/** Holds the engine version. */
FEngineVersionBase ArEngineVer;
/** Holds the engine network protocol version. */
uint32 ArEngineNetVer;
/** Holds the game network protocol version. */
uint32 ArGameNetVer;
};
通过重载operater <<来实现对数据的访问。
UObject的序列化接口
-
void UObject::Serialize( FArchive& Ar )
UObject通过实现Serialize接口来序列化对象数据。
void UObject::Serialize( FArchive& Ar )
{
// These three items are very special items from a serialization standpoint. They aren't actually serialized.
UClass *ObjClass = GetClass();
UObject* LoadOuter = GetOuter();
FName LoadName = GetFName();
// Make sure this object's class's data is loaded.
if(ObjClass->HasAnyFlags(RF_NeedLoad) )
{
Ar.Preload(ObjClass);
// make sure this object's template data is loaded - the only objects
// this should actually affect are those that don't have any defaults
// to serialize. for objects with defaults that actually require loading
// the class default object should be serialized in FLinkerLoad::Preload, before
// we've hit this code.
if ( !HasAnyFlags(RF_ClassDefaultObject) && ObjClass->GetDefaultsCount() > 0 )
{
Ar.Preload(ObjClass->GetDefaultObject());
}
}
// Special info.
if( (!Ar.IsLoading() && !Ar.IsSaving() && !Ar.IsObjectReferenceCollector()) )
{
Ar << LoadName; // 对象名
if(!Ar.IsIgnoringOuterRef())
{
Ar << LoadOuter; // Outer对象
}
if ( !Ar.IsIgnoringClassRef() )
{
Ar << ObjClass; // UClass对象
}
}
// Special support for supporting undo/redo of renaming and changing Archetype. 编辑器中使用
else if( Ar.IsTransacting() )
{
if(!Ar.IsIgnoringOuterRef())
{
if(Ar.IsLoading())
{
Ar << LoadName << LoadOuter;
// If the name we loaded is different from the current one,
// unhash the object, change the name and hash it again.
bool bDifferentName = GetFName() != NAME_None && LoadName != GetFName();
bool bDifferentOuter = LoadOuter != GetOuter();
if ( bDifferentName == true || bDifferentOuter == true )
{
LowLevelRename(LoadName,LoadOuter);
}
}
else
{
Ar << LoadName << LoadOuter;
}
}
}
// Serialize object properties which are defined in the class.
// Handle derived UClass objects (exact UClass objects are native only and shouldn't be touched)
// 判断当前对象是否为UClass的实例, 如果是普通的Object的化就序列化它的所有带UPROPERTY()的成员变量
if (ObjClass != UClass::StaticClass())
{
SerializeScriptProperties(Ar);
}
// Keep track of pending kill
if( Ar.IsTransacting() ) // 编辑中undo/redo时序列化
{
bool WasKill = IsPendingKill();
if( Ar.IsLoading() )
{
Ar << WasKill;
if (WasKill)
{
MarkPendingKill();
}
else
{
ClearPendingKill();
}
}
else if( Ar.IsSaving() )
{
Ar << WasKill;
}
}
// Serialize a GUID if this object has one mapped to it
FLazyObjectPtr::PossiblySerializeObjectGuid(this, Ar);
// Invalidate asset pointer caches when loading a new object
if (Ar.IsLoading() )
{
FStringAssetReference::InvalidateTag();
}
// Memory counting (with proper alignment to match C++)
SIZE_T Size = GetClass()->GetStructureSize();
Ar.CountBytes( Size, Size );
}
-
void UObject::SerializeScriptProperties( FArchive& Ar ) const
该函数用来序列化Object体系类中声明为UPROPERTY()的成员变量。
Paste_Image.png
-
UStruct::SerializeTaggedProperties
序列化对象属性，并且加入tag，这个是为了处理对象类发生变化导致属性匹配不上(版本升级和容错)。
void UStruct::SerializeTaggedProperties(FArchive& Ar, uint8* Data, UStruct* DefaultsStruct, uint8* Defaults, const UObject* BreakRecursionIfFullyLoad) const
{
//SCOPED_LOADTIMER(SerializeTaggedPropertiesTime);
// Determine if this struct supports optional property guid's (UBlueprintGeneratedClasses Only)
const bool bArePropertyGuidsAvailable = (Ar.UE4Ver() >= VER_UE4_PROPERTY_GUID_IN_PROPERTY_TAG) && !FPlatformProperties::RequiresCookedData() && ArePropertyGuidsAvailable();
if( Ar.IsLoading() )
{
// Load tagged properties.
// This code assumes that properties are loaded in the same order they are saved in. This removes a n^2 search
// and makes it an O(n) when properties are saved in the same order as they are loaded (default case). In the
// case that a property was reordered the code falls back to a slower search.
UProperty* Property = PropertyLink;
bool bAdvanceProperty = false;
int32 RemainingArrayDim = Property ? Property->ArrayDim : 0;
// Load all stored properties, potentially skipping unknown ones.
while (1)
{
FPropertyTag Tag;
Ar << Tag; // 读取标记
if( Tag.Name == NAME_None )
{
break;
}
if (!Tag.Name.IsValid())
{
UE_LOG(LogClass, Warning, TEXT("Invalid tag name: struct '%s', archive '%s'"), *GetName(), *Ar.GetArchiveName());
break;
}
// Move to the next property to be serialized
if( bAdvanceProperty && --RemainingArrayDim <= 0 )
{
Property = Property->PropertyLinkNext;
// Skip over properties that don't need to be serialized.
while( Property && !Property->ShouldSerializeValue( Ar ) )
{
Property = Property->PropertyLinkNext;
}
bAdvanceProperty = 0;
RemainingArrayDim = Property ? Property->ArrayDim : 0;
}
// Optionally resolve properties using Guid Property tags in non cooked builds that support it.
if (bArePropertyGuidsAvailable && Tag.HasPropertyGuid)
{
// Use property guids from blueprint generated classes to redirect serialised data.
FName Result = FindPropertyNameFromGuid(Tag.PropertyGuid);
if (Result != NAME_None && Tag.Name != Result)
{
Tag.Name = Result;
}
}
// If this property is not the one we expect (e.g. skipped as it matches the default value), do the brute force search.
if( Property == nullptr || Property->GetFName() != Tag.Name )
{
// No need to check redirects on platforms where everything is cooked. Always check for save games
if ((!FPlatformProperties::RequiresCookedData() || Ar.IsSaveGame()) && !Ar.HasAnyPortFlags(PPF_DuplicateForPIE|PPF_Duplicate))
{
FName EachName = GetFName();
FName PackageName = GetOutermost()->GetFName();
// Search the current class first, then work up the class hierarchy to see if theres a match for our fixup.
UStruct* Owner = GetOwnerStruct();
if( Owner )
{
UStruct* CheckStruct = Owner;
while(CheckStruct)
{
FName NewTagName = UProperty::FindRedirectedPropertyName(CheckStruct, Tag.Name);
if (NewTagName != NAME_None)
{
Tag.Name = NewTagName;
break;
}
CheckStruct = CheckStruct->GetSuperStruct();
}
}
}
UProperty* CurrentProperty = Property;
// Search forward...
for ( ; Property; Property=Property->PropertyLinkNext ) // 查找到对应的Property
{
if( Property->GetFName() == Tag.Name )
{
break;
}
}
// ... and then search from the beginning till we reach the current property if it's not found. 从头再找，这块利用了大部分情况下UProperty是顺序序列化的，提高查找效率。
if( Property == nullptr )
{
for( Property = PropertyLink; Property && Property != CurrentProperty; Property = Property->PropertyLinkNext )
{
if( Property->GetFName() == Tag.Name )
{
break;
}
}
if( Property == CurrentProperty )
{
// Property wasn't found.
Property = nullptr;
}
}
RemainingArrayDim = Property ? Property->ArrayDim : 0;
}
#if WITH_EDITOR
if (!Property)
{
Property = CustomFindProperty(Tag.Name);
}
#endif // WITH_EDITOR
FName PropID = Property ? Property->GetID() : NAME_None;
FName ArrayInnerID = NAME_None;
// Check if this is a struct property and we have a redirector
// No need to check redirects on platforms where everything is cooked. Always check for save games
if (!FPlatformProperties::RequiresCookedData() || Ar.IsSaveGame())
{
if (Tag.Type == NAME_StructProperty && PropID == NAME_StructProperty)
{
const FName NewName = FLinkerLoad::FindNewNameForStruct(Tag.StructName);
const FName StructName = CastChecked<UStructProperty>(Property)->Struct->GetFName();
if (NewName == StructName)
{
Tag.StructName = NewName;
}
}
else if ((PropID == NAME_EnumProperty) && ((Tag.Type == NAME_EnumProperty) || (Tag.Type == NAME_ByteProperty)))
{
const FName NewName = FLinkerLoad::FindNewNameForEnum(Tag.EnumName);
if (!NewName.IsNone())
{
Tag.EnumName = NewName;
}
}
}
const int64 StartOfProperty = Ar.Tell();
if( !Property )
{
//UE_LOG(LogClass, Warning, TEXT("Property %s of %s not found for package: %s"), *Tag.Name.ToString(), *GetFullName(), *Ar.GetArchiveName() );
}
#if WITH_EDITOR
else if (BreakRecursionIfFullyLoad && BreakRecursionIfFullyLoad->HasAllFlags(RF_LoadCompleted))
{
}
#endif // WITH_EDITOR
// editoronly properties should be skipped if we are NOT the editor, or we are
// the editor but are cooking for console (editoronly implies notforconsole)
else if ((Property->PropertyFlags & CPF_EditorOnly) && !FPlatformProperties::HasEditorOnlyData() && !GForceLoadEditorOnly)
{
}
// check for valid array index
else if( Tag.ArrayIndex >= Property->ArrayDim || Tag.ArrayIndex < 0 )
{
UE_LOG(LogClass, Warning, TEXT("Array bound exceeded (var %s=%d, exceeds %s [0-%d] in package: %s"),
*Tag.Name.ToString(), Tag.ArrayIndex, *GetName(), Property->ArrayDim-1, *Ar.GetArchiveName());
}
else if( !Property->ShouldSerializeValue(Ar) )
{
UE_CLOG((Ar.IsPersistent() && FPlatformProperties::RequiresCookedData()), LogClass, Warning, TEXT("Skipping saved property %s of %s since it is no longer serializable for asset: %s. (Maybe resave asset?)"), *Tag.Name.ToString(), *GetName(), *Ar.GetArchiveName() );
}
else if (Property->ConvertFromType(Tag, Ar, Data, DefaultsStruct, bAdvanceProperty))
{
if (bAdvanceProperty)
{
continue;
}
}
else if (Tag.Type != PropID)
{
UE_LOG(LogClass, Warning, TEXT("Type mismatch in %s of %s - Previous (%s) Current(%s) for package: %s"), *Tag.Name.ToString(), *GetName(), *Tag.Type.ToString(), *PropID.ToString(), *Ar.GetArchiveName() );
}
else
{
uint8* DestAddress = Property->ContainerPtrToValuePtr<uint8>(Data, Tag.ArrayIndex);
uint8* DefaultsFromParent = Property->ContainerPtrToValuePtrForDefaults<uint8>(DefaultsStruct, Defaults, Tag.ArrayIndex);
// This property is ok. 读取数据内存
Tag.SerializeTaggedProperty(Ar, Property, DestAddress, DefaultsFromParent);
bAdvanceProperty = true;
if (!Ar.IsCriticalError())
{
continue;
}
}
bAdvanceProperty = false;
// Skip unknown or bad property.
const int64 RemainingSize = Tag.Size - (Ar.Tell() - StartOfProperty);
uint8 B;
for( int64 i=0; i<RemainingSize; i++ )
{
Ar << B;
}
}
}
else
{
check(Ar.IsSaving() || Ar.IsCountingMemory());
UScriptStruct* DefaultsScriptStruct = dynamic_cast<UScriptStruct*>(DefaultsStruct);
/** If true, it means that we want to serialize all properties of this struct if any properties differ from defaults */
bool bUseAtomicSerialization = false;
if (DefaultsScriptStruct)
{
bUseAtomicSerialization = DefaultsScriptStruct->ShouldSerializeAtomically(Ar);
}
// Save tagged properties.
// Iterate over properties in the order they were linked and serialize them.
const FCustomPropertyListNode* CustomPropertyNode = Ar.ArUseCustomPropertyList ? Ar.ArCustomPropertyList : nullptr;
for (UProperty* Property = Ar.ArUseCustomPropertyList ? (CustomPropertyNode ? CustomPropertyNode->Property : nullptr) : PropertyLink;
Property;
Property = Ar.ArUseCustomPropertyList ? FCustomPropertyListNode::GetNextPropertyAndAdvance(CustomPropertyNode) : Property->PropertyLinkNext)
{
if( Property->ShouldSerializeValue(Ar) )// 判断是否要序列化该属性
{
const int32 LoopMin = CustomPropertyNode ? CustomPropertyNode->ArrayIndex : 0;
const int32 LoopMax = CustomPropertyNode ? LoopMin + 1 : Property->ArrayDim;
for( int32 Idx = LoopMin; Idx < LoopMax; Idx++ )
{
uint8* DataPtr = Property->ContainerPtrToValuePtr <uint8>(Data, Idx);
uint8* DefaultValue = Property->ContainerPtrToValuePtrForDefaults<uint8>(DefaultsStruct, Defaults, Idx);
// 判断该属性是否为CDO中的属性值相同，如果相同就不必存了(节约)
if( CustomPropertyNode || !Ar.DoDelta() || Ar.IsTransacting() || (!Defaults && !dynamic_cast<const UClass*>(this)) || !Property->Identical( DataPtr, DefaultValue, Ar.GetPortFlags()) )
{
if (bUseAtomicSerialization)
{
DefaultValue = NULL;
}
#if WITH_EDITOR
static const FName NAME_PropertySerialize = FName(TEXT("PropertySerialize"));
FArchive::FScopeAddDebugData P(Ar, NAME_PropertySerialize);
FArchive::FScopeAddDebugData S(Ar, Property->GetFName());
#endif
FPropertyTag Tag( Ar, Property, Idx, DataPtr, DefaultValue );
// If available use the property guid from BlueprintGeneratedClasses, provided we aren't cooking data.
if (bArePropertyGuidsAvailable && !Ar.IsCooking())
{
const FGuid PropertyGuid = FindPropertyGuidFromName(Tag.Name);
Tag.SetPropertyGuid(PropertyGuid);
}
Ar << Tag;
// need to know how much data this call to SerializeTaggedProperty consumes, so mark where we are
int64 DataOffset = Ar.Tell();
// if using it, save the current custom property list and switch to its sub property list (in case of UStruct serialization)
const FCustomPropertyListNode* SavedCustomPropertyList = nullptr;
if(Ar.ArUseCustomPropertyList && CustomPropertyNode)
{
SavedCustomPropertyList = Ar.ArCustomPropertyList;
Ar.ArCustomPropertyList = CustomPropertyNode->SubPropertyList;
}
Tag.SerializeTaggedProperty( Ar, Property, DataPtr, DefaultValue );
// restore the original custom property list after serializing
if (SavedCustomPropertyList)
{
Ar.ArCustomPropertyList = SavedCustomPropertyList;
}
// set the tag's size
Tag.Size = Ar.Tell() - DataOffset;
if ( Tag.Size > 0 )
{
// mark our current location
DataOffset = Ar.Tell();
// go back and re-serialize the size now that we know it
Ar.Seek(Tag.SizeOffset);
Ar << Tag.Size;
// return to the current location
Ar.Seek(DataOffset);
}
}
}
}
}
static FName Temp(NAME_None);
Ar << Temp;
}
}
-
UStruct::SerializeBinEx
特殊情况才会走这个函数，目前个人认为正常存文件不走这个函数。
void UStruct::SerializeBinEx( FArchive& Ar, void* Data, void const* DefaultData, UStruct* DefaultStruct ) const
{
if ( !DefaultData || !DefaultStruct )
{
SerializeBin(Ar, Data);
return;
}
for( TFieldIterator<UProperty> It(this); It; ++It )
{
// Serializes the property with the struct's data residing in Data, unless it matches the default
// 序列化跟CDO中不一样的属性
It->SerializeNonMatchingBinProperty(Ar, Data, DefaultData, DefaultStruct);
}
}
- UStruct::SerializeBin
//
// Serialize all of the class's data that belongs in a particular
// bin and resides in Data.
//
void UStruct::SerializeBin( FArchive& Ar, void* Data ) const
{
if( Ar.IsObjectReferenceCollector() )
{
for( UProperty* RefLinkProperty=RefLink; RefLinkProperty!=NULL; RefLinkProperty=RefLinkProperty->NextRef )
{
RefLinkProperty->SerializeBinProperty( Ar, Data );
}
}
else if( Ar.ArUseCustomPropertyList )
{
const FCustomPropertyListNode* CustomPropertyList = Ar.ArCustomPropertyList;
for (auto PropertyNode = CustomPropertyList; PropertyNode; PropertyNode = PropertyNode->PropertyListNext)
{
UProperty* Property = PropertyNode->Property;
if( Property )
{
// Temporarily set to the sub property list, in case we're serializing a UStruct property.
Ar.ArCustomPropertyList = PropertyNode->SubPropertyList;
Property->SerializeBinProperty(Ar, Data, PropertyNode->ArrayIndex);
// Restore the original property list.
Ar.ArCustomPropertyList = CustomPropertyList;
}
}
}
else
{
for (UProperty* Property = PropertyLink; Property != NULL; Property = Property->PropertyLinkNext)
{
Property->SerializeBinProperty(Ar, Data);
}
}
}
下面为调试时的几张堆栈图:
Paste_Image.png
Paste_Image.png
uasset文件格式
UE中使用统一的格式存储资源(uasset, umap)，每个uasset对应一个包(package)，存储一个UPackage对象时，会将该包下的所有对象都存到uasset中。UE的uasset文件格式很像Windows下的DLL文件格式(PE格式)，并且使用起来神似(下一节分析Linker)。
UE4文件格式.png
- File Summary 文件头信息
- Name Table 包中对象的名字表
- Import Table 存放被该包中对象引用的其它包中的对象信息(路径名和类型)
- Export Table 该包中的对象信息(路径名和类型)
- Export Objects 所有Export Table中对象的实际数据。
/**
* A "table of contents" for an Unreal package file. Stored at the top of the file.
*/
struct FPackageFileSummary
{
/**
* Magic tag compared against PACKAGE_FILE_TAG to ensure that package is an Unreal package.
*/
int32 Tag;
private:
/* UE4 file version */
int32 FileVersionUE4;
/* Licensee file version */
int32 FileVersionLicenseeUE4;
/* Custom version numbers. Keyed off a unique tag for each custom component. */
FCustomVersionContainer CustomVersionContainer;
public:
/**
* Total size of all information that needs to be read in to create a FLinkerLoad. This includes
* the package file summary, name table and import & export maps.
*/
int32 TotalHeaderSize;
/**
* The flags for the package
*/
uint32 PackageFlags;
/**
* The Generic Browser folder name that this package lives in
*/
FString FolderName;
/**
* Number of names used in this package
*/
int32 NameCount;
/**
* Location into the file on disk for the name data
*/
int32 NameOffset;
/**
* Number of gatherable text data items in this package
*/
int32 GatherableTextDataCount;
/**
* Location into the file on disk for the gatherable text data items
*/
int32 GatherableTextDataOffset;
/**
* Number of exports contained in this package
*/
int32 ExportCount;
/**
* Location into the file on disk for the ExportMap data
*/
int32 ExportOffset;
/**
* Number of imports contained in this package
*/
int32 ImportCount;
/**
* Location into the file on disk for the ImportMap data
*/
int32 ImportOffset;
/**
* Location into the file on disk for the DependsMap data
*/
int32 DependsOffset;
/**
* Number of references contained in this package
*/
int32 StringAssetReferencesCount;
/**
* Location into the file on disk for the string asset references map data
*/
int32 StringAssetReferencesOffset;
/**
* Location into the file on disk for the SearchableNamesMap data
*/
int32 SearchableNamesOffset;
/**
* Thumbnail table offset
*/
int32 ThumbnailTableOffset;
/**
* Current id for this package
*/
FGuid Guid;
/**
* Data about previous versions of this package
*/
TArray<FGenerationInfo> Generations;
/**
* Engine version this package was saved with. For hotfix releases and engine versions which maintain strict binary compatibility with another version, this may differ from CompatibleWithEngineVersion.
*/
FEngineVersion SavedByEngineVersion;
/**
* Engine version this package is compatible with. See SavedByEngineVersion.
*/
FEngineVersion CompatibleWithEngineVersion;
/**
* Flags used to compress the file on save and uncompress on load.
*/
uint32 CompressionFlags;
/**
* Value that is used to determine if the package was saved by Epic (or licensee) or by a modder, etc
*/
uint32 PackageSource;
/**
* Array of compressed chunks in case this package was stored compressed.
*/
TArray<FCompressedChunk> CompressedChunks;
/**
* If true, this file will not be saved with version numbers or was saved without version numbers. In this case they are assumed to be the current version.
* This is only used for full cooks for distribution because it is hard to guarantee correctness
**/
bool bUnversioned;
/**
* Location into the file on disk for the asset registry tag data
*/
int32 AssetRegistryDataOffset;
/** Offset to the location in the file where the bulkdata starts */
int64 BulkDataStartOffset;
/**
* Offset to the location in the file where the FWorldTileInfo data starts
*/
int32 WorldTileInfoDataOffset;
/**
* Streaming install ChunkIDs
*/
TArray<int32> ChunkIDs;
int32 PreloadDependencyCount;
/**
* Location into the file on disk for the preload dependency data
*/
int32 PreloadDependencyOffset;
};
导入表条目FObjectImport
/**
* UObject resource type for objects that are referenced by this package, but contained
* within another package.
*/
struct FObjectImport
{
/**
* The name of the UObject represented by this resource.
* Serialized
*/
FName ObjectName; // 对象名称
/**
* Location of the resource for this resource's Outer. Values of 0 indicate that this resource
* represents a top-level UPackage object (the linker's LinkerRoot).
* Serialized
*/
FPackageIndex OuterIndex; // 对象的Outer的Index
/**
* The name of the package that contains the class of the UObject represented by this resource.
* Serialized
*/
FName ClassPackage; // 该对象的类元数据(UClass对象）所在的包名
/**
* The name of the class for the UObject represented by this resource.
* Serialized
*/
FName ClassName; // 该对象的类元数据名称
// 后面的数据为运行时填写
/**
* The UObject represented by this resource. Assigned the first time CreateImport is called for this import.
* Transient
*/
UObject* XObject;
/**
* The linker that contains the original FObjectExport resource associated with this import.
* Transient
*/
FLinkerLoad* SourceLinker; // 该对象由哪个Linker加载的
/**
* Index into SourceLinker's ExportMap for the export associated with this import's UObject.
* Transient
*/
int32 SourceIndex;
bool bImportPackageHandled;
bool bImportSearchedFor;
bool bImportFailed;
};
导出表的条目FObjectExport
/**
* UObject resource type for objects that are contained within this package and can
* be referenced by other packages.
*/
struct FObjectExport
{
/**
* The name of the UObject represented by this resource.
* Serialized
*/
FName ObjectName; // 对象名称
/**
* Location of the resource for this resource's Outer. Values of 0 indicate that this resource
* represents a top-level UPackage object (the linker's LinkerRoot).
* Serialized
*/
FPackageIndex OuterIndex; // 对象的Outer的Index
/**
* Location of the resource for this export's class (if non-zero). A value of zero
* indicates that this export represents a UClass object; there is no resource for
* this export's class object
* Serialized
*/
FPackageIndex ClassIndex;
/**
* Location of this resource in export map. Used for export fixups while loading packages.
* Value of zero indicates resource is invalid and shouldn't be loaded.
* Not serialized.
*/
FPackageIndex ThisIndex;
/**
* Location of the resource for this export's SuperField (parent). Only valid if
* this export represents a UStruct object. A value of zero indicates that the object
* represented by this export isn't a UStruct-derived object.
* Serialized
*/
FPackageIndex SuperIndex;
/**
* Location of the resource for this export's template/archetypes. Only used
* in the new cooked loader. A value of zero indicates that the value of GetArchetype
* was zero at cook time, which is more or less impossible and checked.
* Serialized
*/
FPackageIndex TemplateIndex; // 对象原形的Index
/**
* The object flags for the UObject represented by this resource. Only flags that
* match the RF_Load combination mask will be loaded from disk and applied to the UObject.
* Serialized
*/
EObjectFlags ObjectFlags;
/**
* The number of bytes to serialize when saving/loading this export's UObject.
* Serialized
*/
int64 SerialSize; //对象占据的磁盘大小
/**
* The location (into the FLinker's underlying file reader archive) of the beginning of the
* data for this export's UObject. Used for verification only.
* Serialized
*/
int64 SerialOffset; // 文件中的偏移
/**
* The location (into the FLinker's underlying file reader archive) of the beginning of the
* portion of this export's data that is serialized using script serialization.
* Transient
*/
int32 ScriptSerializationStartOffset;
/**
* The location (into the FLinker's underlying file reader archive) of the end of the
* portion of this export's data that is serialized using script serialization.
* Transient
*/
int32 ScriptSerializationEndOffset;
/**
* The UObject represented by this export. Assigned the first time CreateExport is called for this export.
* Transient
*/
UObject* Object;
/**
* The index into the FLinker's ExportMap for the next export in the linker's export hash table.
* Transient
*/
int32 HashNext;
/**
* Whether the export was forced into the export table via OBJECTMARK_ForceTagExp.
* Serialized
*/
bool bForcedExport;
/**
* Whether the export should be loaded on clients
* Serialized
*/
bool bNotForClient; // 游戏Client是否使用该对象
/**
* Whether the export should be loaded on servers
* Serialized
*/
bool bNotForServer; // 游戏server是否使用该对象
/**
* Whether the export should be always loaded in editor game
* False means that the object is
* True doesn't means, that the object won't be loaded.
* Serialized
*/
bool bNotAlwaysLoadedForEditorGame;
/**
* True if this export is an asset object.
*/
bool bIsAsset;
/**
* Force this export to not load, it failed because the outer didn't exist.
*/
bool bExportLoadFailed;
/**
* Export is a dynamic type.
*/
enum class EDynamicType : uint8
{
NotDynamicExport,
DynamicType,
ClassDefaultObject,
};
EDynamicType DynamicType;
/**
* Export was filtered out on load
*/
bool bWasFiltered;
/** If this object is a top level package (which must have been forced into the export table via OBJECTMARK_ForceTagExp)
* this is the GUID for the original package file
* Serialized
*/
FGuid PackageGuid;
/** If this object is a top level package (which must have been forced into the export table via OBJECTMARK_ForceTagExp)
* this is the package flags for the original package file
* Serialized
*/
uint32 PackageFlags;
};
备注: FPackageIndex表示Linker中Import Table或Export Table中的索引, 分如下情形
- PackageIndex > 0 表示在Export Table中的索引，实际索引 Index = PackageIndex - 1;
- PackageIndex < 0 表示在Export Table中的索引，实际索引 Index = -(PackageIndex + 1);
- PackageIndex == 0 表示当前UPackage对象
FLinkerLoad
负责将uasset文件中的对象加载到内存中，起桥梁作用。相关源码:
Engine\Source\Runtime\CoreUObject\Public\UObject\LinkerLoad.h
Engine\Source\Runtime\CoreUObject\Private\UObject\LinkerLoad.cpp
-
FLinkerLoad::ELinkerStatus FLinkerLoad::Tick( float InTimeLimit, bool bInUseTimeLimit, bool bInUseFullTimeLimit );
用于解析uasset文件，当bInUseTimeLimit为true时, Tick不会一次性做完解析工作，分时间片进行加载，在一帧里，Tick()不会占用太多时间。 -
UObject* CreateExport( int32 Index );
创建Export Table中Index位置的对象。 -
void LoadAllObjects(bool bForcePreload);
加载Export Table中的所有对象。
在阅读CreateExport, CreateImport等对象加载代码时需要明白Outer, Class, PackageIndex这些概念，先根据类型创建出对象，然后才Serialize对象的数据。
注: 读取包中对象时，可以一次性加载所有export table中的对象，也可以按需加载某个对象(比如包P0中的对象A被包P1引用，在加载P1时，可能只会加载P0中的对象A，而不是P0中的所有对象)。
关于异步加载
Engine\Source\Runtime\CoreUObject\Private\Serialization\AsyncLoading.cpp
Engine\Source\Runtime\CoreUObject\Private\Serialization\AsyncLoadingThread.h
FLinkerSave
负责将内存Package中的对象存储到uasset文件。相关源码:
Engine\Source\Runtime\CoreUObject\Public\UObject\LinkerSave.h
Engine\Source\Runtime\CoreUObject\Private\UObject\LinkerSave.cpp
LinkerSave做的活不多， 要特别注意序列化Object时的巧妙.
FArchive& FLinkerSave::operator<<( UObject*& Obj )
{
FPackageIndex Save;
if (Obj)
{
Save = MapObject(Obj); // 返回的是PackageIndex
}
return *this << Save;
}
在保存一个Package时主要工作是在FSavePackageResultStruct UPackage::Save(UPackage* InOuter, UObject* Base, EObjectFlags TopLevelFlags, const TCHAR* Filename, FOutputDevice* Error, FLinkerLoad* Conform, bool bForceByteSwapping, bool bWarnOfLongFilename, uint32 SaveFlags, const class ITargetPlatform* TargetPlatform, const FDateTime& FinalTimeStamp, bool bSlowTask);函数中。
源码路径:Engine\Source\Runtime\CoreUObject\Private\UObject\SavePackage.cpp
部分值得注意的代码截图:
Paste_Image.png
Paste_Image.png
Paste_Image.png
写入Name Table
Paste_Image.png
Paste_Image.png
写入Export Object
结尾
还有以下几点没有讲述
- uasset文件的压缩
- uasset的Cook(过滤掉与游戏发布时无关的数据)
- BulkData(BulkData是指一大块数据, 它也被存在uasset文件中，但是加载对象的时候可以不加载它，等到需要时在问LinkerLoad要数据，例如UTexture2D的纹理数据的加载)。
