Experience Analytics not working

Hello Sitecorian,

How many of you have encountered an error or the situation where graph on the experience analytics dashboard or graph on launchpad's dashboard is not working

I recently encountered with this scenario where i was presented with the situation where simply graph was not coming up, Before digging completely into it and checking logs and everything, From very first high level thoughts pointed me to check

• Is the valid certificate is used for xconnect?
• Are thumbprints are correct in all configs? We had scaled environment so i needed to check the thumbprint in all the needed roles.
• Is xconnect site is accessible from CD? Without any https exceptions?
• Is XDB is enable? 
• Is Tracking is enable? 

Everything looked ok on first place, so i finally took my sleeves up and started digging into details, Below are the details steps and exceptions those were captured and i will also show you the topology of instances we were using.

Scaled instance

We are using Sitecore 9.0.2 XP1 installation so we had CM and CD two physical servers, and on CM we had all roles installed with different web site in IIS, so basically all roles are working separately.

Now it was time to start to do some initial check, in most 99% cases it is certificate issue or certificate thumbprint issue so i quickly did following

1) Checked xconnect certificate on CM and CD and also double checked the thumbprint used 
You can find Here The detail steps how to see correct certificate and correct thumbprint of that certificate and make sure the same thumbprint is used in all the roles, In our case everything was good, right thumbprint was used in all roles, all had correct thumbprint of xconnect certificate.

2) Checked by visiting the xconnect site URL from CD and CM
To make sure that visits are registered, first step is to check if the xconnect site is accessible from CD instance and it was being accessible in our case, right certificate was used and correct site was being resolved.

3) Very initial steps after sitecore installation is to "Deploy marketing definitions" and "Rebuild link database"
It is must to deploy marketing definitions after configuring the scaled instance, as shown below in the image taken from Sitecore Documentation 
This steps was performed, i restarted IIS and visited the CD site but still there was no graph.
4) Double check the connection string on all roles.
Just to make sure that all roles are having right connection strings and having proper username and password, and right certificate thumbprint.
 5) Check the log of XConnect Instance, CM server Processing instance and Reporting instance
On checking the xconnect log i came across below exception

System.UnauthorizedAccessException: Access to the registry key 'Global' is denied


To fix this exception, we will need to add app pool user to the following groups

Performance Monitor Users
Performance Log Users

to do that, go to Computer Management and go to following path



Now double click both of this and add xconnect app pool use to this group, restart IIS and check the log again, GOOD, the error was gone

NOTE: Make sure to see whichever role is having this exception, add them to this group.

6) Message: Login failed for user 'poolsuser' in processing role instance log.
Because processing does the processing for collected data and processes it, this exception also gave me the clue that something is wrong, and i checked in the database and found out that in connection string for processing this user was used.

So i checked the DB and processing database and found out that this user did not exist for some reason, so created user and gave it proper access rights, restarted the processing instance and this error was gone, and now processing user was able to connect to DB.

7) Dimension could not be resolved for for dimension id: {GUID}
Looking further, when connection strings were checked and all instance were able to connect to DB , i started getting

10472 07:59:51 ERROR [Experience Analytics]: SegmentProcessor failed to process interaction 'fde57c16-f6a0-0000-0000-058df2ab65f2' segment '2985ad79-b475-494d-a3e0-df2573e66f04' - System.InvalidOperationException: Dimension could not be resolved for for dimension id: fa282c5b-a373-4b89-9977-18a8498ee53e
   at Sitecore.ExperienceAnalytics.Aggregation.Data.Model.AggregationSegment.get_Dimension()
   at Sitecore.ExperienceAnalytics.Aggregation.Data.Model.AggregationSegment.GetData(IVisitAggregationContext context)
   at Sitecore.ExperienceAnalytics.Aggregation.Pipeline.SegmentProcessor.ProcessSegments(IInteractionAggregationContext context, IEnumerable`1 segments)
10472 07:59:51 ERROR Aggregation Error
Exception: System.InvalidOperationException
Message: Dimension could not be resolved for for dimension id: fa282c5b-a373-4b89-9977-18a8498ee53e
Source: Sitecore.ExperienceAnalytics.Aggregation
   at Sitecore.ExperienceAnalytics.Aggregation.Pipeline.SegmentProcessor.ProcessSegments(IInteractionAggregationContext context, IEnumerable`1 segments)
   at Sitecore.ExperienceAnalytics.Aggregation.Pipeline.InteractionProcessor.AggregateSegments(IInteractionAggregationContext context, AggregationScope scope)
   at Sitecore.Analytics.Aggregation.Pipeline.InteractionAggregationPipelineProcessor.Process(InteractionAggregationPipelineArgs args) 

I tried to search this in sitecore and found out that, Dimensions did exist on following path

"/sitecore/system/Marketing Control Panel/Experience Analytics/Dimensions/Connect/By contact orders"

We found out that this dimension belongs to Sitecore Commerce and processing instance also need this dimension and not only CM instance

So We went to Sitecore Commerce installation package and found out the folder "Sitecore Commerce ExperienceAnalytics Core 11.2.83.zip" and manually copied DLLs and Configs to processing server's bin and App_Config

Restarted the IIS but no luck, because one final step was missing

8) Rebuild Reporting DB
I tried to look into the table and found out that interaction table was actually having interactions, so only thing was that they were not being aggregated and processed to show the report somehow.

So It was time to Rebuild Reporting DB, Restarted IIS.



And.......Hurrreeeeyyyy the graph was back and started registering the hits, Phew...huge relief.

I had some close interaction with few of my colleague which really helped me understand the flow of information and xconnect troubleshooting

Kiran Patil
Mahek Chaniyara
Sreekrishnan

NOTE: After having this analytics issue, I found out that in most of the cases related to experience analytics and xconnect issue, Following are the most frequent reasons

1) Xconnect certificate and its thumbprint issue
2) Marketing definitions not deployed (Dimensions and Interaction will not be found otherwise)
3) Reporting DB is not rebuild

I hope above steps will help someone troubleshoot the issue related to similar situation




 

Xconnect certificate thumbprint and communication with sitecore and xconnect

Hello People,

One of my blog post i was writing about the issue i had with my xconnect and experience analytics not working

On the same line i had to troubleshoot and check xconnect certificate and thumbprint in config files, So i also discovered in this process that lot of people like me did not have exact idea about how xconnect and sitecore communicates and what is thumbprint and why it is used and where it is configured, So thought to do a quick blog post about it.

Communication between Sitecore and Xconnect
Everyone of us will need to work with xconnect directly or indirectly, so its better to understand how sitecore talks to xconnect.


xconnect (Definition)
xConnect is the service layer that sits in between the xDB and any trusted client, device, or interface that wants to read, write, or search xDB data. Communication must happen over HTTPS and clients must have the appropriate certificate thumbprint

Now my point of interest here is to tell you is how Sitecore and xconnect connects and what is the  thumbprint for it and where it is configured.

So xconnect is the server where client connects, so here in our case sitecore is the client which connect to xconnect for its services like analytics and marketing automation features and services which are abstracted in xconnect.

Why xconnect certificate is used
It provides additional layer of scurity and It is considered secure than traditional user name and password to communicate and it is cryptographically secure way to have a communication between Sitecore & Xconnect.

Now if we observe the xconnect AppSettings.config file of xconnect role

• AllowInvalidClientCertificates 

         This defines if the Sitecore is allowed to connect even if the certificate is invalid, So this is also   interesting to know that if you still want to connect if you do not have trusted client certificate, you can still use that certificate but you just need to keep the value of AllowInvalidClientCertificates =true. (though not recommended for production)

• ValidateCertificateThumbprint

         This is thumbprint of the xconnect certificate, and this setting defines that what other roles should have which thumbprint, you can verify this thumbprint with the certificate thumbprint to make sure that which certificate is being used to communicate.



As shown above, i have opened the xconnect certificate and see the thumbprint, so we can make sure that this is the certificate and the thumbprint should be used in all roles (if scaled environment)

• Thumbprint needed in what roles in scaled environments 

In our case we had CM server and CD server, and on CM server we had different roles installed on same CM server with different IIS instance of those role configured, So go to Connectionstring.config and AppSettings.config of all roles like Processing, Reporting, Xconnect Search, Content Managemnet, Content Delivery and verify the thumbprint with xconnect thumbprint and make sure that those are same as xconnect certificate thumbprint.

So, hope above information helps to understand why xconnect is needed, where to find thumbprint, what if we do not have trusted certificate and what all roles will need the same thumbprint to connect with xconnect.