标签：.net framework linq dataset linq to object
这篇来谈论一下Linq第三个方面的应用：Linq to Object，只要是继承了IEnumerable或IQueryable接口的Object都能使用Linq特性进行操作。在操作过程当中可能很多人都觉得不好调试不能实时地观察结果数据集，想把IQuery的Linq查询语句转换成数据表DataTable，要怎么实现转换呢？来看一下。
先来说一场景解释一下为什么需要用Linq来解决一些问题，能解决一些什么样的问题，相对于SQL，DataTable等一些传统操作方式有哪些优势？
场景：目前主要数据源有多个，而且数据库的类型可能不一样（SQL Server、Oracle等），也可能分布在不同的服务器上，数据量比较大，要操作的表基本都是千万级的数据量，而且不同数据库里面的表需要作Join查询才能得到满足条件的数据。
传统操作方式：
A). 如果同是SQL Server数据库，那么两台数据库之间可以建立Linked Server连接，这样分布在两台或多台不同服务器上的数据库就能满足联合操作的条件。但因为是Linked Server连接，需要网络传输，性能上比同非分布式的操作要差很多。
B). 如果是SQL Server数据库和Oracle数据库，那么两台数据库之间也可以建立Linked Server连接，这样分布在两台或多台不同服务器上的数据库就能满足联合操作的条件。但如果要在SQL Server上访问Oracle数据源该怎么操作？ 有一种方式是：OpenQuery(sql)，这个性能当然也不会好。
上面两种场景虽然都有实现方式，但主要问题是性能，如果是web系统，需要实时反映查询结果该怎么办？很简单，web页面呈现会有时间限制，如果1~2分钟的时间去加载数据，那么后果是什么?页面直接timeout了，就算Ajax，用户能耐心地等待两分钟吗？所以可以排除用这种传统方式的可能性。
替代操作方式 -- 商业智能（BI)：
如果要用BI的话，可以很轻松地解决所有这些问题，但这需要在数据库仓库层面作相应设计，运用定时运行的Job可以定期把不同数据源的数据汇集到查询数据库，然后直接限制用户可访问的数据源，比如限定访问特定时期内的数据，或者提供可预定的历史报表功能。
如果只是系统里面一个小的需求，那么需要重新架构一个复杂的数据仓库吗？答案是：成本过高。
替代的操作方式 -- 操作DataTable 或 Linq操作：
先将所需的数据分别从不同的分布式的数据库里面尽量查询出来，尽量的含义是，利用已经知道的查询条件尽量返回一个最小的结果集，同样还是为了解决性能问题。然后放在DataTable里面，剩下的操作是处理DataTable的问题了。
处理DataTable有两种方式：DataTable数据行循环， Linq。
‘ DataTable数据行循环
For Each row As DataRow In table.Rows
‘ 业务逻辑 row("column name")
Next
‘ --------------------------------
‘ Linq操作, 以前项目里面的一个例子。
‘Load all records from Assets Change tracking table
getAssetChangeTrackingData(searchResultsTable, assetChangeTrackingSearchResultsTable, assetchangeTrackingListForSP, _
assetchangeTrackingListForSP, assetChangeTrackingModsFileOrigin, assetchangetrackingModsEDSAssetID, _
String.Empty, String.Empty, String.Empty, String.Empty, String.Empty)
‘Load 2D History Barcode data and gmassets data
getTwoDHistoryJoiningData(datamartTable, TwoDhistTable, assetchangeTrackingListForSP, SerialNumberList, _
BarcodeList, AssetIDList, RSNList, BidSegmentList)
‘Get finial Binding data table
‘--------------------------------------------------------------------------------------------
Dim searchResult = searchResultsTable.AsEnumerable()
Dim changingTracking = assetChangeTrackingSearchResultsTable.AsEnumerable()
‘Load records from Assets Changing table
Dim finalResult = From search In searchResult _
Join changing In changingTracking _
On search.Field(Of String)("Asset ID") Equals changing.Field(Of String)("Asset ID") _
And search.Field(Of String)("File Origin") Equals changing.Field(Of String)("File Origin") _
Select New With _
{ _
.SerialNumber = search.Field(Of String)("Serial Number") _
, .AssetAction = search.Field(Of String)("Asset Action Taken") _
}
dtFirst = CustomLINQtoDataSetMethods.CopyToDataTable(finalResult)
Dim finalQuery = dtFirst.AsEnumerable()
‘Load records that are newly added from Assets Changing table
Dim searchNoneIn = From search In searchResult _
Where Not (From changing In finalQuery _
Where Not changing.IsNull("SerialNumber") _
And changing.Field(Of String)("AssetAction").ToUpper = "M" _
Select changing.Field(Of String)("SerialNumber")).Contains(search.Field(Of String)("Serial Number") _
) _
And search.Field(Of String)("Asset Action Taken").ToUpper = "A" _
Select New With _
{ _
.SerialNumber = search.Field(Of String)("Serial Number") _
, .BarCode = search.Field(Of String)("BarCode") _
, .AssetTag = search.Field(Of String)("Asset Tag") _
, .MaintenanceVendor = "Unknown" _
, .LeaseEndDate = "0000-00-00" _
, .PONumber = search.Field(Of String)("PO Number") _
, .ManufacturerPartNumber = search.Field(Of String)("Manufacturer Part Number") _
, .MaintenanceEndDate = "0000-00-00" _
, .Manufacturer = search.Field(Of String)("Manufacturer") _
, .Make = search.Field(Of String)("Make") _
, .Model = search.Field(Of String)("Model") _
, .BidSegment = search.Field(Of String)("Bid Segment") _
, .HostName = search.Field(Of String)("Host Name") _
, .InstallDate = search.Field(Of DateTime?)("Install Date").ToString _
, .Floor = search.Field(Of String)("Floor") _
, .Room = search.Field(Of String)("Room") _
, .RSN = search.Field(Of String)("RSN") _
, .BuildingName = search.Field(Of String)("Building Name") _
, .Country = search.Field(Of String)("Country") _
, .ProductType = search.Field(Of String)("Product Type") _
, .FileOrigin = search.Field(Of String)("File Origin") _
, .AssetID = search.Field(Of String)("Asset ID") _
, .ClientCode = search.Field(Of String)("Client Code") _
}
Dim GMAssetsAll = datamartTable.AsEnumerable()
Dim FinalAssetsAll = From AssetsAll In GMAssetsAll _
Select New With _
{ _
.SerialNumber = AssetsAll.Field(Of String)("Serial Number") _
, .BarCode = AssetsAll.Field(Of String)("BarCode") _
, .AssetTag = AssetsAll.Field(Of String)("Asset Tag") _
, .MaintenanceVendor = AssetsAll.Field(Of String)("Maintenance Vendor") _
, .LeaseEndDate = AssetsAll.Field(Of DateTime?)("Lease End Date").ToString _
, .PONumber = AssetsAll.Field(Of String)("PO Number") _
, .ManufacturerPartNumber = AssetsAll.Field(Of String)("Manufacturer Part Number") _
, .MaintenanceEndDate = AssetsAll.Field(Of DateTime?)("Maintenance End Date").ToString _
, .Manufacturer = "Unknown" _
, .Make = AssetsAll.Field(Of String)("Make") _
, .Model = AssetsAll.Field(Of String)("Model") _
, .BidSegment = AssetsAll.Field(Of String)("Bid Segment") _
, .HostName = AssetsAll.Field(Of String)("Host Name") _
, .InstallDate = AssetsAll.Field(Of DateTime?)("Install Date").ToString _
, .Floor = AssetsAll.Field(Of String)("Floor") _
, .Room = AssetsAll.Field(Of String)("Room") _
, .RSN = AssetsAll.Field(Of String)("RSN") _
, .BuildingName = AssetsAll.Field(Of String)("Building Name") _
, .Country = AssetsAll.Field(Of String)("Country") _
, .ProductType = AssetsAll.Field(Of String)("Product Type") _
, .FileOrigin = AssetsAll.Field(Of String)("File Origin") _
, .AssetID = AssetsAll.Field(Of String)("Asset ID") _
, .ClientCode = AssetsAll.Field(Of String)("Client Code") _
}
‘Add any newly added records to Assets All set
Dim BindingTable = FinalAssetsAll.Union(searchNoneIn)
Dim FinalGMAssetsAll As DataTable = CustomLINQtoDataSetMethods.CopyToDataTable(BindingTable)
Dim TwoDHistory = TwoDhistTable.AsEnumerable()
Dim GMAssetsQuery = FinalGMAssetsAll.AsEnumerable()
‘Calculate retagging reports by joining with 2D Barcode History table
Dim RetagginReport = From AssetsAll In GMAssetsQuery Join TwoDHist In TwoDHistory _
On AssetsAll.Field(Of String)("SerialNumber") Equals TwoDHist.Field(Of String)("Old Tag Serial Number") _
Where TwoDHist.Field(Of String)("Old Tag Serial Number") <> AssetsAll.Field(Of String)("SerialNumber") _
Or TwoDHist.Field(Of String)("Old Tag Bid Segment") <> AssetsAll.Field(Of String)("BidSegment") _
Or TwoDHist.Field(Of String)("Old Tag PO Number") <> AssetsAll.Field(Of String)("PONumber") _
Or TwoDHist.Field(Of String)("Old Tag Maintenance Vendor") <> AssetsAll.Field(Of String)("MaintenanceVendor") _
Or TwoDHist.Field(Of String)("Old Tag Manufacturer Part Number") <> AssetsAll.Field(Of String)("ManufacturerPartNumber") _
Or TwoDHist.Field(Of String)("Old Tag Model") <> AssetsAll.Field(Of String)("Model") _
Or TwoDHist.Field(Of DateTime?)("Old Tag Lease End Date").ToString <> AssetsAll.Field(Of String)("LeaseEndDate") _
Or TwoDHist.Field(Of DateTime?)("Old Tag Maintenance End Date").ToString <> AssetsAll.Field(Of String)("MaintenanceEndDate") _
Order By AssetsAll.Field(Of String)("BidSegment"), AssetsAll.Field(Of String)("Country"), _
AssetsAll.Field(Of String)("RSN"), AssetsAll.Field(Of String)("Floor"), AssetsAll.Field(Of String)("Room") _
Select New With _
{ _
.SerialNumber = AssetsAll.Field(Of String)("SerialNumber") _
, .OldTagSerialNumber = TwoDHist.Field(Of String)("Old Tag Serial Number") _
, .BarCode = AssetsAll.Field(Of String)("BarCode") _
, .AssetTag = AssetsAll.Field(Of String)("AssetTag") _
, .OldTagAssetTag = AssetsAll.Field(Of String)("AssetTag") _
, .MaintenanceVendor = AssetsAll.Field(Of String)("MaintenanceVendor") _
, .OldTagMaintenanceVendor = TwoDHist.Field(Of String)("Old Tag Maintenance Vendor") _
, .LeaseEndDate = AssetsAll.Field(Of String)("LeaseEndDate").ToString _
, .OldTagLeaseEndDate = TwoDHist.Field(Of DateTime?)("Old Tag Lease End Date").ToString _
, .PONumber = AssetsAll.Field(Of String)("PONumber") _
, .OldTagPONumber = TwoDHist.Field(Of String)("Old Tag PO Number") _
, .ManufacturerPartNumber = AssetsAll.Field(Of String)("ManufacturerPartNumber") _
, .OldTagManufacturerPartNumber = TwoDHist.Field(Of String)("Old Tag Manufacturer Part Number") _
, .MaintenanceEndDate = AssetsAll.Field(Of String)("MaintenanceEndDate").ToString _
, .OldTagMaintenanceEndDate = TwoDHist.Field(Of DateTime?)("Old Tag Maintenance End Date").ToString _
, .Manufacturer = TwoDHist.Field(Of String)("Manufacturer") _
, .Make = AssetsAll.Field(Of String)("Make") _
, .Model = AssetsAll.Field(Of String)("Model") _
, .OldTagModel = TwoDHist.Field(Of String)("Old Tag Model") _
, .BidSegment = AssetsAll.Field(Of String)("BidSegment") _
, .OldTagBidSegment = TwoDHist.Field(Of String)("Old Tag Bid Segment") _
, .HostName = AssetsAll.Field(Of String)("HostName") _
, .InstallDate = AssetsAll.Field(Of String)("InstallDate").ToString _
, .Floor = AssetsAll.Field(Of String)("Floor") _
, .Room = AssetsAll.Field(Of String)("Room") _
, .RSN = AssetsAll.Field(Of String)("RSN") _
, .BuildingName = AssetsAll.Field(Of String)("BuildingName") _
, .Country = AssetsAll.Field(Of String)("Country") _
, .ProductType = AssetsAll.Field(Of String)("ProductType") _
, .TwoDTagLastPrintDate = TwoDHist.Field(Of DateTime?)("2D Tag Last Print Date").ToString _
, .FileOrigin = AssetsAll.Field(Of String)("FileOrigin") _
, .AssetID = AssetsAll.Field(Of String)("AssetID") _
, .ClientCode = AssetsAll.Field(Of String)("ClientCode") _
}
RecordReturn = CustomLINQtoDataSetMethods.CopyToDataTable(RetagginReport)
----------------------------------------------------------------------------------------------------------------------------------------------------
上面是以前一个项目里面处理的一个报表功能，就是这种场景的应用，每次查询后面都转换了一次，转换成DataTable，这个不是必须的，但调试时更容易看到结果。Linq有延迟查询功能，所以不作任何转换可以直接把所有的IQueryable结合在一起查询，最后用ToList()、Single()等返回真实结果。
来说一下CopyToDataTable，这个在MSDN上面已经有一个扩展方法来处理，而且经过测试使用没有任何问题：
‘‘‘ <summary>
‘‘‘ Convert non-datarow generic type to datatable
‘‘‘ </summary>
‘‘‘ <typeparam name="T">generic object</typeparam>
‘‘‘ <remarks></remarks>
Public Class ObjectShredder(Of T)
‘ Fields
Private _fi As FieldInfo()
Private _ordinalMap As Dictionary(Of String, Integer)
Private _pi As PropertyInfo()
Private _type As Type
‘ Constructor
Public Sub New()
Me._type = GetType(T)
Me._fi = Me._type.GetFields
Me._pi = Me._type.GetProperties
Me._ordinalMap = New Dictionary(Of String, Integer)
End Sub
Public Function ShredObject(ByVal table As DataTable, ByVal instance As T) As Object()
Dim fi As FieldInfo() = Me._fi
Dim pi As PropertyInfo() = Me._pi
If (Not instance.GetType Is GetType(T)) Then
‘ If the instance is derived from T, extend the table schema
‘ and get the properties and fields.
Me.ExtendTable(table, instance.GetType)
fi = instance.GetType.GetFields
pi = instance.GetType.GetProperties
End If
‘ Add the property and field values of the instance to an array.
Dim values As Object() = New Object(table.Columns.Count - 1) {}
Dim f As FieldInfo
For Each f In fi
values(Me._ordinalMap.Item(f.Name)) = f.GetValue(instance)
Next
Dim p As PropertyInfo
For Each p In pi
values(Me._ordinalMap.Item(p.Name)) = p.GetValue(instance, Nothing)
Next
‘ Return the property and field values of the instance.
Return values
End Function
‘‘‘ <summary>
‘‘‘ Loads a DataTable from a sequence of objects.
‘‘‘ </summary>
‘‘‘ <param name="source">The sequence of objects to load into the DataTable.</param>
‘‘‘ <param name="table">The input table. The schema of the table must match that
‘‘‘ the type T. If the table is null, a new table is created
‘‘‘ with a schema created from the public properties and fields of the type T.</param>
‘‘‘ <param name="options">Specifies how values from the source sequence will be applied to
‘‘‘ existing rows in the table.</param>
‘‘‘ <returns> A DataTable created from the source sequence.</returns>
‘‘‘ <remarks></remarks>
Public Function Shred(ByVal source As IEnumerable(Of T), ByVal table As DataTable, ByVal options As LoadOption?) As DataTable
‘ Load the table from the scalar sequence if T is a primitive type.
If GetType(T).IsPrimitive Then
Return Me.ShredPrimitive(source, table, options)
End If
‘ Create a new table if the input table is null.
If (table Is Nothing) Then
table = New DataTable(GetType(T).Name)
End If
‘ Initialize the ordinal map and extend the table schema based on type T.
table = Me.ExtendTable(table, GetType(T))
‘ Enumerate the source sequence and load the object values into rows.
table.BeginLoadData()
Using e As IEnumerator(Of T) = source.GetEnumerator
Do While e.MoveNext
If options.HasValue Then
table.LoadDataRow(Me.ShredObject(table, e.Current), options.Value)
Else
table.LoadDataRow(Me.ShredObject(table, e.Current), True)
End If
Loop
End Using
table.EndLoadData()
‘ Return the table.
Return table
End Function
Public Function ShredPrimitive(ByVal source As IEnumerable(Of T), ByVal table As DataTable, ByVal options As LoadOption?) As DataTable
‘ Create a new table if the input table is null.
If (table Is Nothing) Then
table = New DataTable(GetType(T).Name)
End If
If Not table.Columns.Contains("Value") Then
table.Columns.Add("Value", GetType(T))
End If
‘ Enumerate the source sequence and load the scalar values into rows.
table.BeginLoadData()
Using e As IEnumerator(Of T) = source.GetEnumerator
Dim values As Object() = New Object(table.Columns.Count - 1) {}
Do While e.MoveNext
values(table.Columns.Item("Value").Ordinal) = e.Current
If options.HasValue Then
table.LoadDataRow(values, options.Value)
Else
table.LoadDataRow(values, True)
End If
Loop
End Using
table.EndLoadData()
‘ Return the table.
Return table
End Function
Public Function ExtendTable(ByVal table As DataTable, ByVal type As Type) As DataTable
‘ Extend the table schema if the input table was null or if the value
‘ in the sequence is derived from type T.
Dim f As FieldInfo
Dim p As PropertyInfo
For Each f In type.GetFields
If Not Me._ordinalMap.ContainsKey(f.Name) Then
Dim dc As DataColumn
‘ Add the field as a column in the table if it doesn‘t exist
‘ already.
dc = IIf(table.Columns.Contains(f.Name), table.Columns.Item(f.Name), table.Columns.Add(f.Name, f.FieldType))
‘ Add the field to the ordinal map.
Me._ordinalMap.Add(f.Name, dc.Ordinal)
End If
Next
For Each p In type.GetProperties
If Not Me._ordinalMap.ContainsKey(p.Name) Then
‘ Add the property as a column in the table if it doesn‘t exist
‘ already.
Dim dc As DataColumn
dc = IIf(table.Columns.Contains(p.Name), table.Columns.Item(p.Name), table.Columns.Add(p.Name, p.PropertyType))
‘ Add the property to the ordinal map.
Me._ordinalMap.Add(p.Name, dc.Ordinal)
End If
Next
‘ Return the table.
Return table
End Function
End Class
‘‘‘ <summary>
‘‘‘ Provide public function for calling
‘‘‘ </summary>
‘‘‘ <remarks></remarks>
Public Module CustomLINQtoDataSetMethods
<Extension()> _
Public Function CopyToDataTable(Of T)(ByVal source As IEnumerable(Of T)) As DataTable
Return New ObjectShredder(Of T)().Shred(source, Nothing, Nothing)
End Function
<Extension()> _
Public Function CopyToDataTable(Of T)(ByVal source As IEnumerable(Of T), ByVal table As DataTable, ByVal options As LoadOption?) As DataTable
Return New ObjectShredder(Of T)().Shred(source, table, options)
End Function
End Module
上面就是Linq to Object的一些操作方式，简便，不需要作N多的循环，如同在操作数据库表一样。
Linq技术三：Linq to Object 和生成数据表的扩展方法,布布扣,bubuko.com
Linq技术三：Linq to Object 和生成数据表的扩展方法
标签：.net framework linq dataset linq to object
原文地址：http://blog.csdn.net/cs258475870/article/details/27193273
