free geoip Community Server Migration Script -- Round 2 - Jayson's Blog - jaysonKnight.com
jaysonKnight.com
Welcome to my corner of the internet

Community Server Migration Script -- Round 2

[Update 02/11]  Please see my new post for more details concerning this utility, sorry for all the ping-ponging of posts concerning the migration tool.  The end result will be worth it.  [ /update]

Ok I’ll admit it, my previous post about migrating from .Text to Community Server was a bit arduous/tedious/hacky, so I’ve taken the time to write a proper managed console application that (attempts) to take the some of the pains out of the migration, namely not having to manually update each of the 9 scripts in the Transform Data Tasks (by auto-generating the DTS package for you...so yes, it still uses DTS, I wouldn't have it any other way).  You can download the console application here.  Before running the application, make sure you properly configure the DotTextCS.exe.config file; I’ve commented each key with a brief description, though it should be quite straightforward.  In a future release (soon), I’ll try to get rid of having to manually find your BlogID, UserID PostAuthor, SectionID, and SettingsID…but for now that remains the only manual process.  Once again, to get those values you will need to do the following:

  • For the BlogID setting, look in the blog_Content table in your .Text database.
  • For the other settings, they can all be gathered by looking at the cs_posts table in your Community Server database.

You should start with a fresh install of the Community Server database, then set up your blog(s) and make a test comment (so that the UserID (etc) information is logged to the cs_posts table for harvesting).  After configuring the .config file, run the console application, simply hit ‘y’ to indicate you want to create the package, and then run the package from Enterprise Manager (or you can execute the package directly from the console app, there is a key called ExecutePackage in the .config file…setting this to true sidesteps having to go to Enterprise Manager, though you might get an error or two…they can be ignored).  You will also need to execute the following SQL statements to tidy everything up, I’ll just quote my previous post:

If you have linked back to your own blog, execute the following SQL statement in Query Analyzer to change all the links to your new URL (using your URL’s of course, I’ve left mine in for reference):

DECLARE @originalURL varchar(100)
DECLARE @newURL varchar(100)
SET @originalURL = 'http://jaysonknight.com/blog'
SET @newURL = 'http://jaysonknight.com/cs/blogs/jayblog'
UPDATE    cs_Posts
SET       Body = REPLACE(CAST(Body AS nvarchar(4000)), @originalURL, @newURL),
       FormattedBody = REPLACE(CAST(FormattedBody AS nvarchar(4000)), @originalURL, @newURL)

If you have posts over 4000 chars long, you should be writing novels for a living instead of doing this.  If anyone knows of a work-around for the 4000 char limit, please share.

If you would like to update your hit statistics, use the following SQL statement (replace the db names with the .Text db you’re migrating from):

UPDATE    cs_Posts
SET       TotalViews = [your_db]..blog_EntryViewCount.WebCount
FROM    [your_db]..blog_EntryViewCount
WHERE    cs_Posts.PostID = [your_db]..blog_EntryViewCount.EntryID

UPDATE    cs_Posts
SET       AggViews = [your_db]..blog_EntryViewCount.AggCount
FROM    [your_db]..blog_EntryViewCount
WHERE    cs_Posts.PostID = [your_db]..blog_EntryViewCount.EntryID

I’ll get these dumped in the utility in a future release.  The same previous caveats apply, namely links/post categories don’t sort properly…also, your post categories will also show up as link categories, just delete them using the CS admin page…I haven’t figured out the column mapping for those yet.  I have tested this application on numerous (fresh) Community Server databases and am batting 100%, but let me know how it works out.  If you want the source code, ping me and I’ll happily provide it, and I’ll post it to my site once I get it cleaned up (and when it has all the functionality I want).  This is a huge improvement over my previous solution IMO…but maybe that’s just me.  Good luck, and let me know how it goes.


Posted Feb 08 2005, 04:57 PM by Jayson Knight

41 Comments

TrackBack wrote Migration Script -- .Text to Community Server RC2
on 02-09-2005 1:02 AM
TrackBack wrote re: .Text to CS migration
on 02-09-2005 1:11 AM
Tim Marman wrote re: Community Server Migration Script -- Round 2
on 02-09-2005 1:58 AM
Awesome!

A few questions:

1) Will the utility preserve the post id and original date? (This will ensure that the links don't change)

2) Does this copy comments & trackbacks?

jayson knight wrote re: Community Server Migration Script -- Round 2
on 02-09-2005 2:13 AM
Everything is preserved exactly as it is in .Text; if you'd like to see my migration, head over to http://jaysonknight.com/CS/blogs/ and poke around. You'll see that its virtually identical to my .Text blog (comments, dates, post categories, etc).

For now, this script is a one shot deal (meaning you have to start with a fresh CS db each time), but I'll put some sort of insert functionality into it soon. Make sure you run the SQL statement to update links *if* you linked back to your own blog...otherwise those links will still point to your old site. Let me know how it goes.

Oh, and use a test db first if you can...if not, make sure you back up your db first!!!
Tim Marman wrote re: Community Server Migration Script -- Round 2
on 02-09-2005 8:17 AM
Hmm, it's failing for me.

Type 'y' to build the DotText2CS DTS package, anything else to exit: y

Unhandled Exception: System.InvalidCastException: QueryInterface for interface DTS.CustomTask failed.
at DTS.Task.get_CustomTask()
at DotTextCS.Package.Task_Sub1(Object goPackage)
at DotTextCS.Package.BuildPackage()
at DotTextCS.Main.Process()
at DotTextCS.Main.Main()
jayson knight wrote re: Community Server Migration Script -- Round 2
on 02-09-2005 8:39 AM
Make sure the SQL Server client tools are installed on your machine, otherwise it can't find the DispID for the DTSPackage COM library. I completely forgot that this is necessary, something for me to think about. Thanks for the info.
Martin wrote re: Community Server Migration Script -- Round 2
on 02-09-2005 11:02 AM
Hmm, I do have the SQL server tools installed, but get the same error as Tim

C:\Download\Migrate>dottextcs
Type 'y' to build the DotText to CS DTS package, anything else to exit: y

Unhandled Exception: System.InvalidCastException: QueryInterface for interface D
TS.CustomTask failed.
at DTS.Task.get_CustomTask()
at DotTextCS.Package.Task_Sub1(Object goPackage)
at DotTextCS.Package.BuildPackage()
at DotTextCS.Main.Process()
at DotTextCS.Main.Main()


Is there any other information I can give you to track down what's causing the problem?

Thanks for your efforts anyway - it'll be fab when we get this working!
Tim Marman wrote re: Community Server Migration Script -- Round 2
on 02-09-2005 4:55 PM
Yeah, I have the client tools installed too...

Tim Marman wrote re: Community Server Migration Script -- Round 2
on 02-09-2005 7:32 PM
Do you have to copy the DLL locally or something? Where are you running this from?
jayson knight wrote re: Community Server Migration Script -- Round 2
on 02-09-2005 11:56 PM
I think I know what the issue is; I compiled the app on my local machine (running SQL2k Personal (not MSDE)) with no service packs (DOH!). From a little research I did, SP2 and higher have an interface change in the DTSPackage library (poopoo on MS for that one). I'm installing SP3a and will give it a recompile and we'll see what happens. Thanks for the feedback.
Martin wrote re: Community Server Migration Script -- Round 2
on 02-09-2005 11:59 PM
Sounding promising again - look forward to trying it.
jayson knight wrote re: Community Server Migration Script -- Round 2
on 02-10-2005 2:28 AM
Alrighty, I've recompiled against SQL SP3 (updated the DTS interop assembly). I was able to successfully run the console application as well as the generated DTS package. The ZIP file is at the same location mentioned in the post.

I updated the code to automatically get most of the user info from each database (commented those keys out in the .config file).

Give it another shot and let me know how it goes, the sooner the better as I have some time tonight to work on it. Good luck.
jayson knight wrote re: Community Server Migration Script -- Round 2
on 02-10-2005 2:32 AM
I forgot to mention that there needs to be 1 row of data in the cs_posts table, so simply create a new blog and do a single test post to it, otherwise the app won't be able to get your user info from CS.
Tim Marman wrote re: Community Server Migration Script -- Round 2
on 02-10-2005 3:30 AM
What happens if you specify this stuff? Will it pick it up automatically?

(The problem being that I'm using the same master database for multiple sites, so I want to make sure it's moved specifically into the right CS site).
jayson knight wrote re: Community Server Migration Script -- Round 2
on 02-10-2005 3:46 AM
I see what you're saying. The app picks up the .Text BlogID via the DotTextUsername key specified in the .config file. (maps to the UserName column in bloc_Config). If there is more than one row in blog_Config with the same username, only the last row will be retrieved; for example, if you have two rows with a UserName of foo, with blogIDs of 0 and 1, the value of 1 will be used to retrieve all related posting content. All the tables in .Text key off of the blog ID, so only the relevant content for that blog id is retrieved and dumped into CS. As far as I can tell, you should be able to dump numerous BlogID content into 1 instance of CS via this app, but I haven't tested it (wrote it with a single blog in mind...just for me ;-) ), I say that with about 90% certainty, and I'll give it a shot.

I should probably work some conditional logic into the app to let you specify whether to use the config file for user info, or to use database queries, for now everything is keyed off of the .Text UserName specified in the .config file. I can have an updated build out in a couple of hours. Let me know if you have any other questions, and let me know if that made any sense :-).
jayson knight wrote re: Community Server Migration Script -- Round 2
on 02-10-2005 6:58 AM
There is a new drop available at the link specified in my original post. I updated the utility to address Tim's comment about multiple sites in a .Text DB. The new drop keys off of the UserName column in the blog_Config table for getting the correct content out of .Text, so as long as the UserName is unique, the correct content will be migrated over to CS.

I also updated the utility to let the user decide whether to get user info by letting the app do it for you, or explicitely defining the criteria in the .config file.
Tim Marman wrote re: Community Server Migration Script -- Round 2
on 02-10-2005 7:55 AM
Actually, I was referring to the CS side - I need to specify the right SettingsID on that side... because I'm running multiple communities within the same DB.
Martin wrote re: Community Server Migration Script -- Round 2
on 02-10-2005 1:19 PM
Still getting primary key violation problems running the script - am I doing something wrong? I started from a blank CS database, set up a blog and posted 1 record to find the necessary values, then created and ran the script.

at least it runs now, but the primary key issue seems to be related to the single post I have to make to get the data for the config file.

If I delete the entry from cs_posts the first (biggest) package does run but it then fails on the next on because the tables are not empty - is this a requirement? i.e. should I delete the single post from the admin interface then try again?
jayson knight wrote re: Community Server Migration Script -- Round 2
on 02-11-2005 1:00 AM
@Martin

Yes, the tables need to be blank...sorry that this is still kind of a manual process, I'm working on getting some logic in the utility that gets all of that info for you, but trying to figure out what the telligent folks have done with the CS db has been tedious at best (as is trying to wrap your head around any new project); bear in mind I have no specs/docs/etc, it's basically been hunt and peck. Check out my newest drop, it allows you to let the app choose whether to extrapolate the info from the db, or you can explicitely define the info in the config.

@Tim

I'm still working on getting a multi-user solution into place, though it's not near as trivial as the single blog migration. I have a feeling that the telligent folks might beat me to it. Question: how much of the CS database do you already have set up, i.e. do you have user/blogs/blog owners/etc defined? Or are you starting completely from scratch?

Thanks again for the feedback guys.
Robert McLaws wrote re: Community Server Migration Script -- Round 2
on 02-11-2005 10:41 AM
Jason,

I'm starting with 30 blogs, over 12,000 posts, and a blank CS database. I'd rather not have to set any of the blogs up, but I will if I have to. If this tool can help me convert LonghornBlogs.com in as painless a manner as possible, that would be awesome
Martin wrote re: Community Server Migration Script -- Round 2
on 02-11-2005 1:00 PM
Hmm, I'm getting exceptions trying to create the package in the latest version - it asks the initial questions then fails with
"The application generated an exception that could not be handled" Process id = 0xc18 (3096) Thread id=0xbf0 (3056) and then it terminates.
Martin wrote re: Community Server Migration Script -- Round 2
on 02-11-2005 1:02 PM
Got this error after the exit:

Unhandled Exception: System.InvalidOperationException: Timeout expired. The tim
eout period elapsed prior to obtaining a connection from the pool. This may hav
e occurred because all pooled connections were in use and max pool size was reac
hed.
at System.Data.SqlClient.SqlConnectionPoolManager.GetPooledConnection(SqlConn
ectionString options, Boolean& isInTransaction)
at System.Data.SqlClient.SqlConnection.Open()
at DotTextCS.User.GetSettings()
at DotTextCS.Main.Process()
at DotTextCS.Main.Main(String[] args)
jayson knight wrote re: Community Server Migration Script -- Round 2
on 02-11-2005 11:36 PM
@All

I've posted a new drop of the utility; the ability to manually specify user info has been deprecated in favor of a much more automated approach. Here are the steps you need to take to get everything migrated over at this point:

- Create a new CS database using either the CS installer or the SQL scripts
- Have your users register on CS *with the same email address they use for .Text* (this is crucial as I am now keying everything off of this value). Alternately, you can do this yourself as an admin in the admin section for a trivial amount of users. Regardless, the user info must be in the database before creation of the blogs.
- Here is the only tedious part that still exists; each blog must be manually created, and you must explicitely say who the owner of the blog is in the admin section (the second text box in the 'create blog' section, needs to be the users username...perhaps you can map a user to a specific security role and let them do this themselves? I don't know...). I do know this is still not an ideal situation for a non-trivial number of blogs, but for a handful its not that bad. The next drop of this utility will automate this process so that only the member info needs to be in the database; it will set up the blogs during the migration of the data itself. I should have this done by the end of the weekend, and I believe it's the only outstanding issue with the utility.
- Run the utility (check your connection settings in the .config file first) If a matching email address isn't found in CS (one that matches a member's email in .Text), it will skip that .Text blog and move to the next one. NOTE: The app will ask you if you'd like to save the packages or execute them directly...please save the packages to the server (choose 'n'), I am getting a visual C++ exception when trying to execute the packages directly from the app, which means it's a DTS issue. I'll try and track it down later. Each DTS package will be saved w/ a corresponding BlogID number (let me know if a differentr identifier for each package would be better). All that's left to do is execute the package(s) you wish to migrate over. It's also worth mentioning that Links/Link Categories are still not sorting properly (i.e. alphabetically), but everything else appears to be working fine.

@Robert

As of this drop, each blog would still need to be manually created. I am looking into how to avoid this for a future drop. If you get a chance, could you run the utility to see if the packages get built correctly? The utility might churn for a bit (haven't built any visual indicators as to what's going on yet), but I'd like to see if the app can handle that load. You can just delete the packages when it's done...just make sure they're all there, most appreciative for your time :-).

@Martin

I switched all the data access stuff over to the Microsoft data access app block...give the app another shot and let me know how it goes.

Good luck.
Tim Marman wrote re: Community Server Migration Script -- Round 2
on 02-12-2005 1:27 AM
Jayson:

The specific SITE and blog is empty, but the database itself isn't.

(In other words, they all use the same connect string - same machine, same DB, etc. When I create the new site, I give a different value for application name (ie one is 'dev' and the other is 'mysitename').

Does that make sense?

I'll post a followup later if it's not :)
Martin wrote re: Community Server Migration Script -- Round 2
on 02-12-2005 11:51 AM
Now it runs without error, says it has created a package, but I can't find it anywhere - also tried executing the package from the app but no results either.

Any ideas?
Martin wrote re: Community Server Migration Script -- Round 2
on 02-12-2005 12:01 PM
Ignore me - being stupid (hey it's early for a Saturday!!) - I'd not created the empty blog first.

It works!! It's transferred all my entries (500+) and comments too - very pleased - THANKS!!
Brian Desmond wrote re: Community Server Migration Script -- Round 2
on 02-13-2005 11:29 PM
Having the same issue - it ran, but, no sign fo the package.
jayson knight wrote re: Community Server Migration Script -- Round 2
on 02-14-2005 12:03 AM
@Brian

Make sure you refresh the "Local Packages" repository; it should be in there.
TrackBack wrote Community Server Migration Script -- Round 3
on 02-15-2005 8:33 AM
Alexandru Minza wrote re: Community Server Migration Script -- Round 2
on 02-20-2005 4:49 PM
It would be great if it would be possible to import all the picture galleries from .Text into CS too :)
jayson knight wrote re: Community Server Migration Script -- Round 2
on 02-21-2005 7:24 AM
@ Alexandru

I will look into this after I get the 1.0 release of CS-.Text converter out...I should be able to write a utility that migrates the picture categories themselves over to CS, but the actual pictures are another story. I won't bore you w/ technical details (unless you like that sort of stuff), but it's not as straightforward as migrating the other stuff over.
Andrei wrote re: Community Server Migration Script -- Round 2
on 02-22-2005 10:35 AM
I're read around ... I guess all 3 of your posts concerning blogs migration. I'm not sure I'm getting this right. Is it necessary or not to post a first blog entry (so that the user info gets into cs_Posts) ?
jayson knight wrote re: Community Server Migration Script -- Round 2
on 02-22-2005 10:44 AM
@ Andrei

Sorry for all the confusion regarding my posts on this tool...it has (rapidly) progressed from a personal project into something better suited for the community as a whole.

As of the most current release, you do not need to post an entry to any of the blogs; all you need to do is set up the users in CS (and they need to have the same email address in CS as in .Text, I'm keying everything off of this so this is crucial), and then set up corresponding blogs for each user (there is a field for the blog owner, this needs to have the correct username specified in the setup of the users mentioned above). After doing this, the utility should build the correct packages for each blog to be migrated over.

This is still a bit of a manual process at this point, but should be fine for a trivial number of blogs. V 1.0 of this tool should be released in a couple of days which will alleviate all of the "manual-ness" of setting up users/blogs. Let me know how it works out for you.
jayson knight wrote re: Community Server Migration Script -- Round 2
on 02-27-2005 7:22 AM
Please see my latest post (and comments) here: http://jaysonknight.com/blog/archive/2005/02/25/1074.aspx
(e)Mail Insecurity wrote An exhaustive list of Community Server resources
on 04-08-2005 12:05 PM
(e)Mail Insecurity wrote An exhaustive list of Community Server resources
on 04-08-2005 12:09 PM
水村部落 wrote An exhaustive list of Community Server resources
on 05-07-2005 5:50 PM
[Editor: This list is a work-in-progress. Please feel free to drop me a note at deving@3sharp.com if...
木可-奔跑的蜗牛 wrote An exhaustive list of Community Server resources
on 06-13-2005 11:31 AM
原文地址:
http://blogs.3sharp.com/Blog/deving/articles/274.aspx

Documentation, Tutorials, and References...
木可-奔跑的蜗牛 wrote An exhaustive list of Community Server resources
on 06-20-2005 5:18 PM
Community Server resources
Carsinigin wrote I need to upgrade
on 04-07-2006 3:27 PM
How To Build a Fireplace wrote How To Build a Fireplace
on 09-16-2008 7:55 AM

“ The potential applications of artificial electromagnetic fields are wide- ranging and can be used in many military or quasi military situations. . . . Some of these potential uses include dealing with terrorist groups, crowd control, controlling breaches

Add a Comment

(required)  
(optional)
(required)  
Remember Me?