Common mistakes in SharePoint 2013 Arhitecture

I am participating in the upgrade from SharePoint 2007 to  SharePoint 2013 on premises. It is nice to be back in upgrade work since it was my main activity last 6 years). This time it's a little bit different.
First of all, the upgrade is from SharePoint 2007 to SharePoint 2013. We do it through a transitional SharePoint 2010 farm.
And second of all, I wasn't involved in the architecture phase of building the farm. I jumped on the upgrade bandwagon just before the final prod upgrade. So, what I do mainly is  verifying that new farm is in the working conditions and testing once again the content migration.
Along the way I see how people built the farm and streamlined the upgrade process. It's a good experience to have to observe different style of working with SharePoint. I am free of making architectural decisions, but at the same time I am experienced enough to see outcome of decisions that were made by others.
This post is about my observations on common SharePoint 2013 architectural mistakes.

I can get "editing in the browser" feature with no additional configuration
No, you have to have 2 conditions to have Office Web Apps on the farm:
1. Office Web App server
2.  Claim based web application

You need to have OWA server. You can't place OWA on App server. It should be a dedicated server for Office Web App server.Plan for an additional server in the farm.
Office Web Apps can be used only by SharePoint 2013 web applications that use claims-based authentication


OWA server will handle all Excel calculation

Office Web Apps Server enables you to view workbooks that contain Data Models that use native data. However, you can’t explore data in items such as PivotTable reports, PivotChart reports, and timeline controls that use a Data Model as the data source.

(Overview of business intelligence capabilities in Office Web Apps Server and SharePoint Server)

Overview of Excel Services in SharePoint Server 2013

Excel Web App runs in one of two modes:

SharePoint view mode   In this mode, Excel Services is used to view workbooks in the browser.

Office Web Apps Server view mode   In this mode, Excel Web App is used to view workbooks in the browser.

Comparing Excel Online, Excel Services, and Excel Web App

       Excel Services, and Excel Web App all have a lot in common, but they are not exactly the same. These applications can differ in what workbook features are supported for viewing in a browser

More on SharePoint 2013 architectural pain points

SP farm doesn't use SQL alias

 An old and common mistake. I have just recently jumped on the project after the farm was already configured. The first thing that makes me sick in the farm, that there is no alias for SQL.

So, in case you sql server dies and need to make sure that you name the new server the same way as previous one.

SharePoint Online migration: a quick analysis of 3rd party tools for migration

Recently I have finished manual migration to SharePoint Online. 

See SharePoint Online migration: Manual migration. Why, How and what are the problems of manual migration? for details.

And this a quick post is my first glance analysis of leading 3rd party tools for migration:

Note: I am not affiliated to any 3rd party SharePoint migration tools. This analysis done in order to choose (if necessary) tools for my migrations projects.

1. Quest Migration Suite for SharePoint - Quest’s Migration Suite for SharePoint is the complete solution for simplifying SharePoint migrations whether to SharePoint on-premises or SharePoint Online. It virtually eliminates the risks of downtime and data loss. With a single, agentless tool requiring just one install, you can seamlessly move your entire SharePoint environment, as well as Microsoft Exchange Public Folders, and Windows files, to on-premises versions of SharePoint, SharePoint Online, or a hybrid of the two. And if you decide to wait to migrate to SharePoint Online down the road, you’ll already have a tool in place to proceed immediately.

   I used the trial version to trim the content before the actual migration. 

Pro:  I liked the easy installation and easy manipulation with SP objects. I have used extensively In-place tagging. I love that they offer a trial version with full functionality. The only limit is the expiration date. 

Cons: UI is ugly. 

 Price: Unknown.

2. Metagolix Content Matrix .

   I haven't tried it, but I have seen a live demonstration and got a chance to ask questions.

Pro:  The product has a feature of comparison of the objects and manipulation (Incrementation update\migrate capabilities)

It seems that it is more robust than Quest.

I like that they have online version ( for SPOnline farms) of their product for control and governance - ControlPoint.

Cons: I haven't got a price sheet after presentation. The whole deal with hiding price is a big annoyance.

 Price: Unknown.

3. AvePoint Content Manager .

    I haven't tried it, but I have seen a live demonstration and got a change to ask questions.

 Pro: A BIG BIG PRO - THE TRUST. AvePoint VP - Jeremy Thake. He is a well known expert in SharePoint world and has  lots of intelligent and hot topic articles on SharePoint. Because of his involvement in the product I feel a huge inclination to trust these guys.

Prices are not hidden.

Cons: The concept of DocAve 6 Platform is complicated in my opinion. I couldn't figured out which product do I need for migration until I have seen a live presentation.

Price: Content Manager - 995 per SP server (web-front, app server)

SharePoint 2013: Architectural changes. Pain Points

As I have launched a migration project to SharePoint 2013, I have started analyzing the architectural changes in SharePoint 2013 farm and other related products.

My goal is to migrate SharePoint 2010 with Project Server 2010 to SharePoint 2013 with Project Server 2013.

Here is my earliest discoveries:

1. Hardware recommendations differ for SharePoint 2013 and for Project Server 2013. But Project Server lives inside the SP Farm.
For small SP farm, you have to have around 12 Gb for Front End and 8 Gb for SQL.
But in case you want to include Project Server in the farm, MS recommends 16Gb for Front End and 16 Gb for Sql.
NOTE: This 16 Gb recommendation doesn't cover additional need for memory by other SP services:

The minimum hardware requirements in this section are recommended in which only the required services to run Project Server 2013 are enabled. Be aware that enabling additional SharePoint Server 2013 features in the farm may require more resources. 

So, it seems like MS intends to sale lots of  Azure VM services!
At this moment, I am struggling to find on-prem VMs for a new demanding farm. Realistically, I will not get more than 10 GB RAM for SP+Project Server 2013.


PAINT POINT: You have to boost hardware not only for SP2013 ,but for Project Sever 2013 accommodation.


2.Office Web Apps server (or OWA farm) should be installed on dedicated server (NOT ON THE SP SERVER).
 And by the way, have you acquainted with PowerShell? OWA Sever doesn't have UI to set settings, you have to use PowerShell 3.0.
  Once you have installed OWA farm (server) , you have to bind (SPWopiBinding) SP to it.

Now, think twice in case of migration.
Not only you have to boost hardware for serves, you have to find an additional server in order to have Office Web Apps. In SP2010 Office Web Apps product was supposed to be installed on the SP server.

PAINT POINT: You have to find an additional server for OWA to provide  functionality that was  "hardware-strain" free in the previous SP version.

3. Web analytics services are removed from Sp2013 as a separate services. Now it's a part of Search Service.

When you upgrade to SharePoint 2013, do not attach and upgrade the databases that contain the data from Web Analytics in SharePoint Server 2010. We recommend that you turn off Web Analytics in the SharePoint Server 2010 environment before you copy the content databases that you want to upgrade to SharePoint 2013.

PAINT POINT: Web analytics are best service to clean your farm before you go to SharePoint 2013. It allows you find unused spwebs. But you have to turn the service off when you want to copy your content db for test upgrade. So, you need to clean the environment (with help of Web analytics) before even your first test upgrade.

4. A new Access Service requires SQL 2012.

 In case you want\or have to provide Access Service in your SP 2013 Farm, you have point the service to SQL 2012 Server.

PAINT POINT: For Access Service you have to have SQL 2012. So if you have planned to stick with SQL 2008, make sure you don't need to have Access Service. In case you need it you have 2 choices: 1. Have 2012 SQL server installed at the first place, or 2. extend you farm and include additional 2012 SQL server to accommodate Access Service Application.

SharePoint 2010 to 2013 Upgrade: Phase I "Cleaning". Unused spweb

As you may know Microsoft recommends to clean the farm first before migrating to a new version of SharePoint. (Clean up an environment before an upgrade to SharePoint 2013)

For my migration story SharePoint 2007 to SharePoint 2010, please refer to "SharePoint 2007 to 2010 Upgrade" online project


Now, I m in the process of migration SharePoint 2010 to SharePoint 2013.
I recommend to start cleaning from top to bottom.
First, check unused web applications - SharePoint 2010 to 2013 Upgrade: Phase I "Cleaning". Unused web applications

Then, go ahead and check unused site collection - SharePoint 2010: Confirm site use and deletion. How to restore Site Collection ?

And finally, let's check what sites (spweb) are not in use

I took following principle to identify unused sites - if the site doesn't have content except the standard one ( ex: Master Page Gallery, one item in Announcements), it's considered an unused site.

To check the content , or to say more specifically, the itemcount in every list in the site, I use this PowerShell  script:

$rootSite = Get-SPSite  {site collection url}                                                                                                                                                                                                     
$rootSite.AllWebs | foreach {  $_.Lists | Select ParentWebUrl, Title, ItemCount |Sort ItemCount -Descending } |Export-Csv webs_content.csv

Once I have got data in the the file, I have created a simple pivot table to check sum of the items per site:

That gives me to determine which site doesn't contain user data (in my case the site the contains only 4 items:

default Announcement, and 3 items in the Master Page Gallery) 

And finally, delete unused sites:

$rootSite = Get-SPSite  {site collection url}                                                                                                                                                                                                       
$sitesToDelete = "{spweb relative url}";

foreach ($site in $sitesToDelete)
{
  $spweb = $rootSite.AllWebs[$site];
  $spweb.Name
  $spweb.Lists| group ItemCount -AsHashTable
  $spweb.Delete()
  $spweb.Dispose()
  "Web is deleted"
}

Test-SpContentDB MissingFeature, MissingSetupFile, MissingWebPart, MissingAssembly. And not so Fabulous 40 templates.

I am on verge of finally get my farm migrated to SP2013 on-prem.
I was suffering with cleaning customization by not so fabulous for me  "Fabulous 40  templates".

I ended up transferring content from customized sites to OTB sites and deleting sites based on the custom site definition.
Then I de-activated features and de-installed custom wsp.

Once I have done that part, I ran test-spcontentdb on my current prod SP2010. To my surprise I got a lots of errors:
MissingFeature, 
MissingSetupFile, 
MissingWebPart, 
MissingAssembly

I have started analyzing the log and discovered that all missing customization are part of Fab 40!

Btw, Fab 40 has a lots of features that start with 75a0fea7 (Here is a quick and dirty look on customization by MS - List of Features with GUIDs)

Now, this post is intended to help some unfortunate souls who suffer from such behavior.

Obvious question in this situation: How can I get rid of MissingFeatures, MissingSetupFile, MissingWebPart, MissingAssembly?

Here is what I did:

1. I have started with Stsadm -o enumsites -includefeatures to find missing features and remove them via  SharePoint Feature Administration and Clean Up Tool

2. I have run following PowerShell commands that have mentioned : Removing [MissingFeature] Database & [MissingWebPart] WebPart class errors from SharePoint 2010

    These PS commands helped me with most MissingFeatures. But MissingSetupFiles, MissingWebParts, MissingAssemblies errors were still in place.

Then I moved to another step of cleansing:

3.I had to clean SQL tables directly based on the following article: Fixing Test-SPContentDatabase warnings

In my case sp object were already deleted, but for some reason sql tables still have rows with no corresponded SharePoint objects. So, I had to delete records directly in SQL.

I know, I know, it's not recommended, but I didn't get another option....

4. Still got MissingAssembly. So, I used this script: REMOVE ORPHAN EVENT HANDLERS IN SHAREPOINT LISTS. 

And once again, in my case I didn't find correspoded to SQL record SharePoint object and had to delete records directly from EventReceivers table.

SharePoint 2010 to 2013 Upgrade: Service Accounts

As you may know there is no in-place upgrade for SharePoint 2013.
You have to build a new SP2013 farm and then bring your content from SP2010 over.

One of the first steps of the upgrade is to plan service accounts that you will use in the new SP2013 farm.
Here is an excellent article by Todd Klindt that will help you with that: Service Account Suggestions for SharePoint 2013

And another excellent and short list of recommended service accounts by Margriet Bruggeman: SharePoint 2013 Best Practices: Service Accounts

Here is an original recommendation from Microsoft:
 Initial deployment administrative and service accounts in SharePoint 2013
 Plan for administrative and service accounts in SharePoint 2013

And a final touch: How to create a SharePoint 2010 admin account and stop using sp_farm

SharePoint 2010: Confirm site use and deletion. How to restore Site Collection ?

One big part of  the migration to a new version of SharePoint is a preparation phase.
This phase emphasizes cleaning or pruning the environment before moving content database (  in case of SharePoint 2013 Upgrade you don't have in-place upgrade option) to a new version.

I recommend to clean from top to bottom.
First, identify unused web applications. (SP Migration: Phase I "Cleaning". Unused web applications )
Then check unused site collections.
At this step, I recommend to make use of "Confirm site use and deletion"

Of course, you can still use this option even you are not planning the upgrade but still feel urge to get of rid of "dead" structure and content.

I found a useful article that explain in detail how this feature works - A Closer Look At “Site use confirmation and deletion”…
As this post mentioned, it's good to know that :

As of the June 2012 CU for SharePoint 2010, the Dead Site Delete timer job now calls proc_DeleteSiteAsync which is explained below - this is good news, and now means that sites deleted by site use conformation and deletion will be subject to the site Recycle Bin.

Let's explore June 2012 CU (build 14.0.6123.5002):
It has resolved issue 2598348 Description of the SharePoint Foundation 2010 hotfix package (Wss-x-none.msp): July 2, 2012

From http://support.microsoft.com/kb/2598348

If a site is deleted by a dead site delete timer job, the site is deleted permanently. Instead, the site should be sent to the Site Collection Recycle Bin.

So, based on this knowledge you can now check 'Automatically delete the site collection if use is not confirmed' with less struggles in your mind.

And btw, Where is Site Collection bin? Here is the asnwer:
SharePoint 2010: SP1 Site Collection Recycle Bin (en-US)

SharePoint 2010 to 2013 Upgrade: Phase I "Cleaning". Unused web applications

As you may know Microsoft recommends to clean the farm first before migrating to a new version of SharePoint. (Clean up an environment before an upgrade to SharePoint 2013)
For my migration story SharePoint 2007 to SharePoint 2010, please refer to "SharePoint 2007 to 2010 Upgrade" online project

Now, I m in the process of migration SharePoint 2010 to SharePoint 2013.

I recommend to start cleaning from top to bottom.

First, check unused web applications

Web analytics reports are really good help for that.

It shows you: Total Number of Page Views, Total Number of Daily Unique Visitors.

Based on that information, it's fairly easy to find abandoned web applications.

So, next step is to stop the related application pool and notify users.