Showing posts with label populate. Show all posts
Showing posts with label populate. Show all posts

Thursday, March 29, 2012

ETL : rows with Errors

I'm using a "Execute SQL Task" to run a stored procedure to populate a Fact table in our dimensional datawarehouse. There is one row which violates a foreign key constraint, which causes the entire task to fail so zero rows are loaded. Is there any way to grab the offending row and send it off to some holding ground and go ahead and load the rest of the rows.

I'm using Execute SQL Task mostly because I am very comfortable with writing SQL whereas the rest of SSIS is a bit unfamiliar to me, but I'm guessing that to handle error rows I might have to change to a different kind of task ?

Thanks

Richard

The best way to do this is to use a lookup in the dataflow to determine if the incoming record violates foreign key rules.

As far as your stored procedure goes, head over to the Transact-SQL forum for help with that.

Tuesday, March 27, 2012

Estimate time to populate full text index

Can someone give me an estimate time to populate a full-
text index, SQL Server 2000:
-14 million rows,
- each row is about 1 paragraph long. (approx 400 words and is stored in a LONG TEXT field)
- Language of the text: mixed.
- W.H.:a single CPU 2.8GHZ with 2GB Ram and have 300GB disk space
-incremental.
it has been running for 3 days now and who knows where we are...
Any way to speed up..
Thanks
mustafa,
Given your hardware configuration, and a table with 14 million rows will
take a substitational amount of time, approx 5 to 8 days.
Your best bet is to attempt to stop the Full or Incremental population and
consider horzatioanlly partitioning the table into several smaller tables,
perhaps broken up by date or PK range. If you plan to continue this, I'd
highly recommend that you consider upgrading all aspects of your hardware.
Specifically, consider a multiple CPU server with 2 or more GB of RAM with
your FT Catalogs on a separate disk controller and disk array configured as
RAID0 or RAID10. You should also set the resource_usage level of the
MSSearch service to 5 via sp_fulltext_service 'resource_usage', 5. However,
this will not help you now. I'd also recommend that you review the SQL
Server 2000 BOL title "Full-text Search Recommendations" and the following
FT Deployment white paper: INF: SQL Server 2000 Full-Text Search Deployment
White Paper at:
http://support.microsoft.com/default...b;en-us;323739
Regards,
John
"mustafa jarrar" <anonymous@.discussions.microsoft.com> wrote in message
news:6EC893FD-DD03-4FA1-BE01-052A19EB03AB@.microsoft.com...
> Can someone give me an estimate time to populate a full-
> text index, SQL Server 2000:
> -14 million rows,
> - each row is about 1 paragraph long. (approx 400 words and is stored in a
LONG TEXT field)
> - Language of the text: mixed.
> - W.H.:a single CPU 2.8GHZ with 2GB Ram and have 300GB disk space
> -incremental.
> it has been running for 3 days now and who knows where we are...
> Any way to speed up..
> Thanks
>