Failure to Provision (nooooooo)

Strange events today, and I am a little lost right now.

Sometime over the last two weeks (probably since my VM was switched from one machine to another), site provisioning has failed on my VM.

In this scenario I am creating a new PWA site called Bibblywibbly.  I want this to be a brand new site with clean DBs.  This fails because of mismatching DBs but if i look back through the logs i see:

- Site created

- DBs Created

- Provisioning schema against Published DB begins

- timeouts for 8sec

- Provisioning kicks off a new correlation with same settings

- reuses site

- job fails at DB check

ULS history is here:

PRovisioning Correlation begins:

Starting ProjectPSISharedApplicationTimerJob.Execute for ‘Project Server Service Application’.

Expanded path to C:\Program Files\Common Files and checked it successfully

Baseboard Manufacturer: ‘Intel Corporation’

SQM: Static lib get app data folder

SQM: Finding files like     ‘C:\Program Files\Microsoft Office Servers\14.0\Logs\Project\data??????????.dat’.

SQM: No files to upload.

SQM: Calculating file statistics for upload throttling

SQM: Static lib get app data folder

SQM: Finding files like     ‘C:\Program Files\Microsoft Office Servers\14.0\Logs\Project\data??????????.dat’.

SQM: SqmUploadAllFiles exiting (0x2260)

Provisioning ‘bibblywibbly’: Beginning ‘Create’ operation for PWA instance.

Adding event 6421 (category: Provisioning, product: Project Server) to spam monitoring list

Received SQM session notification

Starting provisioning of site ‘bibblywibbly’.

Adding event 6954 (category: Provisioning, product: Project Server) to spam monitoring list

Creating site for ‘bibblywibbly’.

Adding event 6945 (category: Provisioning, product: Project Server) to spam monitoring list

Reusing http://project.contoso.com/bibblywibbly site for provisioning bibblywibbly

Adding event 8101 (category: Provisioning, product: Project Server) to spam monitoring list

Provisioning ‘bibblywibbly’: Starting database provisioning.

Adding event 6956 (category: Provisioning, product: Project Server) to spam monitoring list

ConnectionString: ‘Data Source=demo2010a;Initial Catalog=master;Integrated Security=True;Pooling=False;Asynchronous Processing=False;Connect Timeout=15;Application Name=”Microsoft Project Server”’    ConnectionState: Closed ConnectionTimeout: 15

SQL Server ‘demo2010a’ version is ‘10.50.1600.1’.

Provisioning ‘bibblywibbly’: Server versions: Primary 10.50.1600.1, Reporting 10.50.1600.1.

Adding event 6974 (category: Provisioning, product: Project Server) to spam monitoring list

Provisioning ‘bibblywibbly’: Databases specified for site ‘ProjectServer’ are as follows: Published: Name = ‘bibblywibbly_Published’, State = NotCreated Draft: Name = ‘bibblywibbly_Draft’, State = NotCreated Archive: Name = ‘bibblywibbly_Archive’, State = NotCreated Reporting: Name = ‘bibblywibbly_Reporting’, State = NotCreatedAdding event 6975 (category: Provisioning, product: Project Server) to spam monitoring listConnectionString: ‘Data Source=demo2010a;Initial Catalog=master;Integrated Security=True;Pooling=False;Asynchronous Processing=False;Connect Timeout=15;Application Name=”Microsoft Project Server”’    ConnectionState: Closed ConnectionTimeout: 15Creating database ‘demo2010a.bibblywibbly_Published’.ConnectionString: ‘Data Source=demo2010a;Initial Catalog=master;Integrated Security=True;Pooling=False;Asynchronous Processing=False;Connect Timeout=15;Application Name=”Microsoft Project Server”’    ConnectionState: Closed ConnectionTimeout: 15Created database ‘demo2010a.bibblywibbly_Published’.Creating database ‘demo2010a.bibblywibbly_Draft’.ConnectionString: ‘Data Source=demo2010a;Initial Catalog=master;Integrated Security=True;Pooling=False;Asynchronous Processing=False;Connect Timeout=15;Application Name=”Microsoft Project Server”’    ConnectionState: Closed ConnectionTimeout: 15Created database ‘demo2010a.bibblywibbly_Draft’.Creating database ‘demo2010a.bibblywibbly_Archive’.ConnectionString: ‘Data Source=demo2010a;Initial Catalog=master;Integrated Security=True;Pooling=False;Asynchronous Processing=False;Connect Timeout=15;Application Name=”Microsoft Project Server”’    ConnectionState: Closed ConnectionTimeout: 15Created database ‘demo2010a.bibblywibbly_Archive’.Creating database ‘demo2010a.bibblywibbly_Reporting’.ConnectionString: ‘Data Source=demo2010a;Initial Catalog=master;Integrated Security=True;Pooling=False;Asynchronous Processing=False;Connect Timeout=15;Application Name=”Microsoft Project Server”’    ConnectionState: Closed ConnectionTimeout: 15Created database ‘demo2010a.bibblywibbly_Reporting’.Provisioning database demo2010a.bibblywibbly_Published as Published.
you can see above that hte provisioniong job has just created the DBs and has applied the schema to PUBLISHED DB
NOW THIS IS WHERE IT ALL GOES BADConnectionString: ‘Data Source=demo2010a;Initial Catalog=bibblywibbly_Published;Integrated Security=True;Pooling=False;Asynchronous Processing=False;Connect Timeout=15;Application Name=”Microsoft Project Server”’    ConnectionState: Closed ConnectionTimeout: 15
repeats for 8 seconds then a new correlation kicks in
Provisioning ‘bibblywibbly’: Beginning ‘Create’ operation for PWA instance.
leading to
Provisioning ‘bibblywibbly’: One or more of the databases already contains schema. When editing or creating a Project Server instance, you may specify: * Four databases that do not exist * Four existing, blank databases * Four existing Project Server databases of the same version from the same installation. Combinations of blank, new, and existing databases are not allowed.Adding event 7070 (category: Provisioning, product: Project Server) to spam monitoring listProvisioning ‘bibblywibbly’: Failed to provision databases. An exception occurred: Provisioning failed because the databases specified did not match..Adding event 6993 (category: Provisioning, product: Project Server) to spam monitoring listProvisioning ‘bibblywibbly’: Database provisioning failed.Adding event 6958 (category: Provisioning, product: Project Server) to spam monitoring listFailed to provision site bibblywibbly with error: Microsoft.Office.Project.Server.Administration.ProvisionException: Failed to provision databases. —-> Microsoft.Office.Project.Server.Administration.ProvisionException: Provisioning failed because the databases specified did not match.     at Microsoft.Office.Project.Server.Administration.PsiServiceApplication.EnsureDatabases(ProjectProvisionSettings provset, SPSite pwaSite, String adminName, String adminEmail, ProjectDatabaseStateType& originalDatabaseState, Guid& adminGuid)     —- End of inner exception stack trace —-     at Microsoft.Office.Project.Server.Administration.PsiServiceApplication.EnsureDatabases(ProjectProvisionSettings provset, SPSite pwaSite, String adminName, String adminEmail, ProjectDatabaseStateType& originalDatabase…