Zero to Hero - A real life RCA of exact issue in Sitecore Managed Cloud environment

Hello All,

The purpose of today's post is to share a real life burning and escalated scenario which was new to me and how did I approach it and how big the escalations were and what was the outcome

Sitecore's goodwill was at stack not because Sitecore is not capable of handling it but just because our environment was Sitecore Managed Cloud, and any issue that comes if its infra, back end code, front end code will be first pointed as Sitecore issue and that is where our consultancy and experience will play a role to prove that it is not Sitecore issue. 

Issue we faced

Out of the blue our site started giving "504 Gateway Time-out", and it was reported that almost everyone is getting this error, but when we used to browse the site, everything looked good and never 504.

504 Gateway Time-out error tells that, That the request went to Content Delivery servers of Sitecore from gateway, but gateway did not get response in time from those CDs and hence it gave time out error.

One thing we knew was, there is something wrong with backend, but backend can be Sitecore, databases, Solr, Azure itself, Backend can be anything which is involved in processing the requests/response, so we knew our scope is very broader, felt like "Finding a needle from the haystack"

But the side effect of this was customer was loosing on MAU (Monthly Active Users) on which their business model is working and it is the most important KPI for them.

 

Troubleshooting without initial clue

We had to start somewhere, because we had no clue why for certain user this issue is coming, so first thing that we do is to check Sitecore logs to see what happened, because this happened for other users and we were just informed.

Just to mention, we are on Sitecore 10.2 XM on manage cloud set up.

1) Sitecore Logs

We could not find any errors in Sitecore logs which could lead to anything like this, there was no specific pin pointing happening from the logs.

(Later on, we found out that because Sitecore never gave the response back to gateway in order so that something can be logged in the logs, because gateway timed out before the response.)

2) Azure application insights

Because we did not find anything in Sitecore logs, we moved to application insights and used transaction search, where you can just type any string and it will search in logs, and it gave us something

What we found were lot of SOLREXCEPTIONs in the time period where this 504 was reported, from that we found that there is an issue with SOLR whenever 504 happens, so we went to solr and checked the logs 

SOLR was choking when these 504 scenarios were happening

We had little clue now what is happening but then we investigated further graphs of response time and server errors during that time of SOLR and here is what we found

Avg. Response Time

Clearly, there was so many exceptions which we were seeing in LOGs were also observed in this graph too.

But our question was, Though SOLR has less power but why it works almost all the time and what happens in the specific scenario that it chocks up? 

So, we set up some theory around the scenario after getting on a call with marketing team and understanding the scenarios of what exactly they are doing and happening.

More from application insights

We observed that in specific time duration when this happens, All the request/response takes huge time to load and we tried to see in performance tab of AVG. time a request is taking in that specific time duration and what it reveled was also something pointing to SOLR

We observed that some of the layout services queries were taking huge time to get back the response, and all of our components read data from SOLR, so there is possibility that if SOLR chokes, it will take time to respond back.

Now if any request is taking more than 1 min. the gateway is bound to time out and give 504 Gateway-timeout error.

Now, we had to find what is causing this, Because if we hit the same page, it works just fine and in that time frame request finishes in perfect time, So we had to drill down to find if it's really a code issue or a caching issue where requests are being made to server all the time and timing out? 

4) Sitecore Ticket

For safer side we also created a SC ticket with these dumps, in initial findings it was also showing long running queries and choking queries of layout services, and sometimes it showed one component being slow and sometimes it showed other components being slow, which was quite confusing

and memory and CPU analysis kept sending us to BOX-1 of slow running layout queries which were timing out the request, but now we knew that these could be side effect of the main cause of some kind of automatic traffic hitting the site which will choke any server.

5) Memory Dump & CPU analysis

We also enabled auto heal and memory dump and site slow down reports etc. now we had memory dumps collected, but all the dumps were showing layout service choking which we already knew.

6) Rendering caching & Performance enhancement & Cache tuning

Because the reports were also showing that layout service is taking time, we had to dig into every rendering, review implementation of third-party calls, APIM calls, DB audit etc. to make sure nothing is miss configured and which could lead to this

Because this was already escalated and P1 issue, if we do this audit and performance enhancement, it will take time, so we decided to remove some of the rendering to see what happens, even after removal of some of the renderings which were depending on third party API, issue still came very next evening

Now, we had to find a way to first know the scenario of what exactly is happening, so we decided to talk to the team who were reporting this issue and we found out that they are observing whenever they are sending out marketing communication with website link, at that time only this issue is occurring

We even increased DefaultHTMLCache and other cache values as we had enough memory to cache things, but no luck with that.

We knew that those many users are not clicking the link but still our site is showing lot of traffic, so something is happening when SMS are sent, and it is not Sitecore or a backend issue from configuration or code perspective.

So, we looked at it from a fresh perspective and set up our hypothesis. 

Setting up hypothesis around the inputs from marketing team.

We now set the theory, of this is happening when there are lot SMS being sent, because marketing team is sending lot of WhatsApp and SMS comms and it seemed that exactly at the same time site is getting outages

All of the timings matched too, whenever site gave 504, all of the time marketing team sent communications

So, we had now something to think about, because on the website we already did the load testing and server scaling etc. was in place, we were sure we had a good set up which could handle the traffic.

7) Load Testing

Though we decided to do a load testing again, and our load testing was showing all results ok with the same URLs which marketing team was sending in whatsapp or SMS marketing.

So again, we were sure that, it's not about the legitimate traffic but it is something to do with SMS and without clicking also burst of traffic is coming like DDoS

8) Server scaling

We decided to put more power on CDs, so we introduced 4 additional instances just to observe for couple of days so we can take out some reports and observe what is happening.

Even if we did that, we observed same amount of 504 request, and we observed following which was pointing to the same scenario of traffic in few seconds.

 

9) Azure Front Door logs

One more reporting and azure diagnostic query we run to check what is coming on AFD and what is giving 504, we took that report out in excel and something we observed 

We observed that, there were some IP ranges which were hitting the sites in few seconds, so if you observe the above graph too

Within seconds site got 4-5 thousand hits, this behavior is not of users, there will be delay of few seconds, here are the IPs which we found 

• 64.233.173.0/24

• 66.102.6.0/24

• 66.102.7.0/24

• 66.249.82.0/24

• 66.249.83.0/24

• 66.249.84.0/24

• 66.249.88.0/24

• 192.178.11.0/24

• 74.125.215.0/24

Majority of them are google bots, but question was why those many requests are coming? and this was happening only when communications were being sent directly to user's mobile as SMS.

10) Further research yielded interesting facts

We went back to our SMS sending partner who sends out bulk SMS and asked them for a report in those dates of outages, and they also sent out circular that, they are seeing sudden high traffic spikes exactly at the same time when SMS are sent, and those can't be users.

Our research on AI platforms and google search also revealed this behavior of pre-fetching of the link to the site for some security and safe browsing link scanners services reasons, and also, we got this from our partners 

"Recent updates in Google Messages introduced enhanced security scanning and automatic link previews. These previews are generated by background preview agents that fetch the URL in advance to show a link preview to users.

Since these fetches imitate real user behaviour, they were being counted as actual clicks, resulting in inflated click numbers and sudden traffic spikes on your website.
 
Google uses a set of proxy/crawler IPs (including the ranges you shared) to pre-fetch URLs found in SMS messages. Because of this, you may observe:
• Multiple requests to your URLs within 2–3 seconds
• IPs from Google subnets appearing as traffic sources
• Sudden server load spikes right after SMS campaigns
• “Bot clicks” recorded even before users interact"

Wow, we were thrilled to learn this behavior, we could understand google bot hitting sites but not aware about this auto clicking of URLs imitating user click behavior, where traffic looks legitimate 

and the hypothesis we established seemed to have a proven theory ahead of us.

Solution

As we know that actual cause was something else and layout choking, SOLR choking and whole infrastructure getting choked was just side effect of these BOT auto clicks and sending huge traffic.

1) We blocked following user agents along with above IP ranges on WAF.

·       Mediapartners-Google

·       Googlebot

·       AdsBot-Google

·       Applebot

·       HeadlessChrome

·       YandexBot

·       facebookexternalhit

·       facebookcatalog

·       WhatsApp

·       YandexImageResizer

·       YandexMobileBot

·       YandexImages

·       YandexAccessibilityBot

·       YandexRenderResourcesBot

·       YandexUserproxy

·       GoogleMessages

·       okhttp

·       /web/snippet/

·       http4s-blaze

·       SkypeUriPreview Preview/0.5

·       Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/139.0.0.0 Safari/537.36

It has been a week, and everything seems to be working fine, and we can see on AFD reports that those IPs are now getting blocked and now all the legitimate requests are turning into 200 status code.

We double checked this behavior with Sitecore and they gave following links, Which is already offered by Sitecore Managed Cloud for DDoS kind of attack or traffic.

How-to's - Sitecore Managed Cloud – DDoS attack mitigation steps

Support Information - Sitecore Managed Cloud Standard (MCS) PaaS 1.0 — DDoS IP Protection

Next action item is to put a rate limit rule based on the campaign URLs to make sure site does not get overwhelmed as these IP ranges may change and user agent could change too. 

Summary Points

Because when you are working as a Sitecore partner, for customer it's a Sitecore platform, and goodwill of Sitecore goes into stake, in this specific scenario same happened, just because it is on Sitecore Managed Cloud.

But finally, we were able to pull that off and customer was convinced that it is not Sitecore issue :) 

Most important things we learn are from actual pressure situations, and backing ourselves in our intuitions which comes from experience, especially when you have not come across situation like these, here are some take away I will always call out below important points when approaching these kinds of situations 

1) Holding your ground

2) Believe in your hypothesis

3) Never give up 

4) Use your intuitions which comes from experiences 

5) Find alternatives 

6) Start with fresh ideas

7) Take a break to get more fresh ideas

8) Think out of the box

9) Work as a team, to have more brain working

10) Don't leave any stone unturned 

Thank you Manglesh Vyas for being there when needed the AFD reports or azure graphs etc. and my backend mate Kiran Sawant who always made sure that if we want to send any change of backend be it caching or fine-tuning resolvers was sent and checked during this P1 issue.

 

Sitecore - How to show a new marketing promotional page on the same URL as existing home page

Hi Team,

Today i will share one of the solution that we did for one of our customer, I am sure you will or you already might have came across such requirements and found your self in multiple option/solutions and trying to find best suited one for your customer, here is the story and solutions we thought of and finally selecting one out of it which was the best in all scenarios

Also the solution was required in time sensitive deadline before their social marketing campaign begins so we had to come up with the solution and implement and go live before it.

Customer Requirement

They were doing a brand refresh, so whole site supposed to be revamped, With new user interface and UX, but that is a longer route, by the time we create that fully new site for them, they wanted to have a teaser home page, or a new home page to be shown just to give the visitor a feel of what is coming and they can market it using social campaigns.

So their need was, Whenever users visit a website (www.blahblah.com for example), Instead of old home page, it should show a new home page with new logo, new menus and new everything, and from there clicking on a "Go to website" button they can go to old website.

Looks fairly simple task right? But now it came with following caveats 

1) URL should not be changed, so no physical routes, but it should still load the new home page, Still preserving the old home page as we had to send user to old home page from that new marketing page.

2) Once user click on the "Go to website" from new marketing page, User should go to old home page without changing the URL again, and after that user should be in the old pages only, all the navigation should work "as-is"

3) Whenever they hit the site in the browser, always it should first load the new teaser (new home page)

4) From old site logo and all the links which takes user to a home page, should load the old home page only, because now user is into the old site.

5) From email campaign links or social media links or any links via search engine which points to website's home page, should first load the new teaser page (home page).

6) Site is multilingual, so new marketing page also had that language switcher and once language is changed the same marketing page should remain.

Possible solutions with their limitations

Because there are multiple ways of doing one thing, and we started thinking about possible approaches to this requirement, and we also noted down the drawback that came with each approach.

1) Change the startItem to new page

One of the first approach was to change the startItem of site from Sitecore site settings, But because this approach was every time loading the new home page, and to load the old home, page we needed to use the url like www.blahblah.com/home as a physical route, because now Sitecore is not treating this old home as start item, to access this item we had to give the physical path in the URL, this looked ok initially, that on the "Go to home" CTA on a new teaser page we can add www.blahblah.com/home route and it will load the old home page, But because this startItem is being referenced on Sitecore's link generator, all the links of the site needed to be changed or custom URL resolver needed to be written which can transform www.blahblah.com/home/about-us to simply have www.blahblah.com/about-us, and this approach we put on the shelve. 

2) Use redirects from old to new and new to old

Our customer's infra team said, They will use redirects from Azure Front Door to decided when to load which page, But this approach we clearly cancelled with reasoning that, we clearly need some identifiers to decide when to load which page and it can not be done without changing lot more from infra side, so it was also put on shelve.

3)  URL Referrer approach 

Our internal links when clicked will have referrers, our JS can identify if there is referrer and if its our internal page then allow them to go to old home page, and when there is no referrer or a refer which is not the current site then send them to new teaser page , but flow here was it will have flicker because JS can only be executed once page is loaded and it will not have the user experience we wanted.

4) UTM query string approach 

It had flow same as point-3, but then we also thought about reading these on our resolvers and then remove the query string before sending user to a page, otherwise all pages will have those UTM SOURCE attached, Also it might end up in too many redirect if not handled properly, Clearly we thought there can be better approach.

5) Use Personalization 

We were looking for a fast and efficient way to achieve this behavior, Though it can be achieved with personalization but lot of existing components needed to be changed and we wanted to stay away from changing existing components and let them work and behave as-is.

After all these thought process, we clearly knew, we need two things

1) Some how we need to know the way to send user to old and new page using some mechanism.

2) To do point-1 we needed identifier or a decision maker when to do that redirect

Final & Working Solution

ItemResolver with referrer as the identifier

 public class HomeItemResolvercs : HttpRequestProcessor

    {
        public override void Process(HttpRequestArgs args)
        {
            string referrerUrl = System.Web.HttpContext.Current.Request.UrlReferrer?.AbsoluteUri;
			// this is my site name in IIS, if you have it differently hosted, just change this to yours
            string mySiteUrl = "homepagepoc.localhost"; 
            // When path is of root and there is no referrer that means its organically hitting the site home page
            if(string.IsNullOrEmpty(referrerUrl) && System.Web.HttpContext.Current.Request.FilePath.Equals("/"))
            {
                var teaserItem = GetTeaserItem();
                if (teaserItem != null)
                    Context.Item = teaserItem;
            }
           // when path is of root and when there is a referrer that means, home link was clicked from somewhere 
           else if(!string.IsNullOrEmpty(referrerUrl) && System.Web.HttpContext.Current.Request.FilePath.Equals("/"))
            {
              // if from search engine or email campaigns, if referrer comes then also it should load teaser page, for our internal pages, it should always open old home page
               if(!referrerUrl.Contains("localhost") && !referrerUrl.Contains(mySiteUrl))
                {
                    var teaserItem = GetTeaserItem();
                    if (teaserItem != null)
                        Context.Item = teaserItem;
                }
               // for all other links load home page
                var homeItem = GetHomeItem();
                if (homeItem != null)
                    Context.Item = homeItem;
            }
           
        }

        private Item GetTeaserItem()
        {
            var sitecoreDB = Sitecore.Context.Database;
            var sitecoreQuery = $"/sitecore/content/Home/newhome";
            if (sitecoreDB != null)
            {
                var teaserItem = sitecoreDB.SelectSingleItem(sitecoreQuery);
                return teaserItem;
            }
            return null;
        }

        private Item GetHomeItem()
        {
            var sitecoreDB = Sitecore.Context.Database;
            var sitecoreQuery = $"/sitecore/content/Home";
            if (sitecoreDB != null)
            {
                var homeItem = sitecoreDB.SelectSingleItem(sitecoreQuery);
                return homeItem;
            }
            return null;
        }
    }

Small Explanation

Basically, The custom resolver has two main condition and checks 

1) It only executes if the request is for home page, So that other requested routes of other pages will load "as-is" without any issue

2) Because the code only executes when request is made for a home page, now the decision to load either teaser page (new home page) or old home depends on the referrer, 

3) From internal site links, if you want to open the new home page from any link, for example press release or something, you can simply add rel="noreferrer" (we used this for language switcher so that it loads the marketing page only but with new selected language)  to your anchor tags and it will load the teaser page, so you are in control.

When the page is directly hit in a browser, it does not have a referrer, and when someone clicks internal link or "Go to home" link on the page, It will automatically attach the referrer and that is our identifier to decide if we should load the new home page or the old home page, See the demo in action below

So, all organic clicks from search engine/ emails / direct browser url will always load the new teaser page and else all other links of the site, internal links will always load the old home page

Cheers !!! Small but useful solution :) 

NOTE: Don't forget to add the processor for this custom pipeline :), to help you I have put together the code and processor on the GITHUB link   

PS: My colleague Nelson   also worked on couple of other approaches and did POC on his end which reads the custom query string and when user is redirected to the page, those query string will be removed.

Sitecore Stream - Platform DxP Evolution

Hi Team,

I am absolutely Sitecore Stream enthusiast, and if you also keep following what is new in it and explore it features, This blog post is for you

Since it was announced in symposium 2024 and probably the first platform DxP version package was released in FEB-2025 and we are now in Sep end and there have been significant roll outs and we are into 6th release of the platform DxP package.

The purpose of this blog post is to summarize all improvements which came along with different releases for all of you and community, If you have not explored all of the packages.

NOTE: Remember Stream is "AI capabilities throughout the composable stack, be it Content Hub, XMC, CDP, Personalized but this blog post is only for platform DxP package for on-prem XP/XM.

If you see the image above, i have tried to show the package rollout on the timeline as an when they were released and watchout for next package rollout, For other useful features :) 

Important improvements

1) Less configurations and less manual config changes, 

Only changing connection string will do rest, where in initial versions you had to change both in connection string and Sitecore.AIClient.config file too.

It was little lengthy process where you had to

1) Find your organization id, Brand kit id

2) Update them in Sitecore.AiClient.Config

3) Update the connection string

But now with latest version, you just use connection string and provide client_id and client_secret generated from portal and you will be able to use Stream with available brand kits.

2) Metadata extraction from an image 

Now, you can have metadata like Title, Description, Alt text, Keywords auto populated by AI metadata extraction, This was released in latest 1.4.54 package which was not available before, Some example of how efficiently it works is below

For example, I am uploading below image in my media library, For your context, This image is a screenshot of one of my podcast on Sitecore TECH Talks, and if you see how intelligently AI is able to extract its metadata is amazing

Here is the image i uploaded, where Varun and i are having a podcast

Below shows when i uploaded it to a media library and when i click on "Extract" button on the toolbar, How it extracts the metadata using AI.

After selecting a brand kit, You can click next and it will automatically generate metadata for the image.

And it correctly identified that,

1) There are two people engaged in a discussion

2) Event is "Sitecore TECH Talks"

3) It also identified that we are in a modern office 

4) Identified that, There are colorful featuring hanging banners in a backdrop.

Based on above it created Title, Description, Alt, Keywords fields, Pretty cool !!!

3) Availability of multiple brand kits 

Initially only one brand was supported and had to be mapped in config, but with 1.1.63 version onwards, Multiple brand key selection was allowed (of course one brand at a time)

In older versions, You had access to only one brand kit, and that also needed to be configured into Sitecore.AiClient.Config like giving these two settings

<setting name="Sitecore.AiClient.Stream.BrandKitId" value="<<brand kit id>>" />

<setting name="Sitecore.AiClient.Stream.BrandKitReferencePath" value="/api/brands/v1/organizations/org_<<org. id>>/brandkits/<<brand kit id>>" />

but now only connection string is needed and it will automatically allow you to select from multiple brands

4) AI-Assisted generation of translated text 

Language translation was not available which was introduced from 1.2.45 onwards, Where user can select the content item, its fields to translate, the target language, then trigger the AI-assisted generation of translated text. Users can also improve content in an existing language by generating a new version of the same language, I think this is a leap in content generation where language translation is available OOTB and co-pilot and agents can work hand in hand, where content author is in control of the end results, Because translation will just translate content and then content authors can proof read and finalize and publish the content.

5) Content Variants

From 1.1.63 package onwards, Now we are able to generate variants of the content using AI, So for example you can either enter your prompt, or select from existing prompt and AI will generate content in 5 different variants, and you can select the best suited for you and proof read it and save it.

6) AI-assisted content extraction

This is my fav. and huge leap in how we have been using Sitecore as CMS, You can now actually extract content from different sources and put them into your Sitecore item, It was introduced in 1.3.23 package release.

Basically, You can let AI extract content from 

1) File - Allowed file extensions are .jpg, .jpeg, .png, .webp, .txt, or .pdf

2) URL - You can give any files available on internet or your asset management or content hub, Basically any public URL and it will extract the content from it, Only above file extensions are allowed.

3) Text : Just paste a raw text and it will do the extraction.

With content extraction, It not only extracts but also creates an item based on selected template and fill those data in.

Way to go #SitecoreStream team, I keep my self aligned with new rollouts to get my hands onto latest developments

I hope this article will give other people highlights of what have been released so far.

I have some webinars on Stream which i took in past, Do have a look for better understanding of what is stream and what it can do from marketers perspective and from content author perspectives too

Pasting the links here so you can go and check them out

Part-1 : Ideation to Execution, Do more in era of less

Part-2: Configuration to Consumption

Podcast - Stream basics for everyone

NOTE: By the time you read this blog sometime in the future, There could be a new version released, So this blog was published on 30th Sep-2025, So don't get confuse if you happen to read in future where Stream already have new versions released.

Reference - Sitecore Stream

Sitecore XP 10.3 - Some installation errors which i encountered for the first time

Hi Team,

This one will be very quick blog post around some of the issues i encountered during Sitecore 10.3 installation, Specially around SOLR

I have not got these issues in previous version and these were something new i encountered so drafting a quick post around it if it helps some of you save some time.

Also surprisingly, did not get this in error in later versions too like 10.4 

Below are the errors which i encountered 

1) Certificate Chain Was Issued by an Authority That is Not Trusted 

I never had this issue before, As generating certificate is backed in the script, but i still got this, so did little research around it and i found some Stack Exchange discussion, but i followed the same steps and it got resolved for me too, Though they are for older versions

Basically, I had to modify xconnect-xp0.json file, Search "InvokeSqlcmd" in it add "TrustServerCertificate": true to its "Params" property. You will find multiple matches for "InvokeSqlCmd", Do the same change for all of it.

2) Unable to connect to a remote server

Another one, which i was getting was unable to connect to a remote server, Well my internet was working fine, everything was ok, But was again not sure why it is not able to connect to SOLR to download the SOLR package for installation

What worked for me was

I turned off firewall security just for this installation and it worked, there could be better ways, but this worked for me and i was able to install it successfully 

You can tryother alternatives given here

Thank you

Sitecore Stream - Manage brand kits and brand documents and ingest using REST API

    

Hello Sitecore Enthusiastic,

Today i will be sharing one of the recent rollouts from Sitecore Stream which is the developer APIs using which one can directly manage their brand kit, upload documents to it and also start ingestion process and that all using APIs.

As recent as June, Sitecore announced REST APIs for Sitecore Partners using which Sitecore partners can now use Stream REST APIs to integrate with their existing systems and support brand governance.

List of APIs for Sitecore Stream

https://api-docs.sitecore.com/?product=AI+capabilities

1) Brand Management API

    As we know Brand kit is the major building block of how Stream gets its brand awareness from, So using this brand management API, You can create new brand kit, you can get specific brand kit or list of brand kits and also can get specific section of brand kit like brand context, tone of voice etc and also can update them.

2) Document Management API

    Each brand document can represent brand's history, context, tone of voice or what not, So once the brand kit is created, One can now use this document management API to create brand document and upload it to brand kit from where it can be ingested, Use case could be a healthcare client want to have approval workflow before AI starts grounding it, so you can hook this API after certain workflow or approved file is dropped to certain location and use this API to then upload it to brand kit, so the process can be automated before AI starts grounding it, it will be one more additional intelligent step to from compliance or governance point of view.

3) Pipeline API

    During the brand ingestion process, the system analyzes each uploaded document page by page, extracts brand information, and breaks it down into smaller pieces called knowledge chunks. These chunks are then organized, vectorized, and stored as your brand knowledge.

The AI uses this brand knowledge to generate content for each section of the brand kit. After the sections are populated, both the comprehensive brand knowledge and the completed brand kit sections are ready for use by Sitecore copilots and other AI tools.

Prerequisites

Before even using the APIs, you will need to generate JWT token, and to generate the token you will first need to create the Brand kit keys (formerly known as "partner keys") which is typically a client id and secret so that using which you can generate the token and use that token to make subsequent calls.

Sitecore has amazing postman like functionality available right into Sitecore Stream documentation which you can find here Try Now

  curl -X POST 'https://auth.sitecorecloud.io/oauth/token' \

  --header 'Content-Type: application/x-www-form-urlencoded' \

  --data-urlencode 'client_id={YOUR_API_KEY}' \

  --data-urlencode 'client_secret={YOUR_API_SECRET}' \

  --data-urlencode 'grant_type=client_credentials' \

  --data-urlencode 'audience=https://api.sitecorecloud.io'

If you copy above and just import into postman as a raw text and then just given client id and secret, and you can generate the partner client id and client secret using partner keys.

NOTE: Make sure to copy generated ClientId and Secret and keep it somewhere safe, As once generated and dialog box is closed, You can not get those same things back (Though you can generate new)

HINT : Do you know what is the difference between xp/xm keys and brand kit keys? Both have different scopes, XP/XM keys does not give you access to brand kit APIs, so if you want to play around with brand kit APIs, Make sure you generate brand kit keys.

I have my sample created and imported into postman and it generated token like below 

Now, subsequent calls you can try out from Sitecore portal itself, Some of the calls are below, where i am trying to get list of brand kits i have 

Similarly you can try other calls and make an application of your own which could communicate with Sitecore Stream and get the things done, I have defined use cases which you can do.

It is important to know which region your stream app is operating because while making a call you will need to determine the server url or endpoint which will be validated against your actual region, If not done correctly you might get into errors (Explained below)

Errors

When playing around with APIs, you might get below exceptions 

1) 'str' object has no attribute 'HTTP_400_BAD_REQUEST'

This happens typically when server URL is wrong against which you are making a request, Your stream app will be hosted on SaaS and it will have its region, As of now there four types of server avaialble

a) Production West Europe Server which has "https://ai-brands-api-euw.sitecorecloud.io" URL.

b) Production East US API Server which has "https://ai-brands-api-use.sitecorecloud.io" URL

c) Production Australia East API Server which has "https://ai-brands-api-aue.sitecorecloud.io" URL

d) Production Japan East API Server which has "https://ai-brands-api-jpe.sitecorecloud.io" URL

2) The request failed with status code: 403 - Invalid organizationId

This happens where organizationId is wrong, Typically when you are using Sitecore's UI or "Try now" button, it gives you sample data for region specific env variables, you will need to change those variables to match with your org id etc. else this error will come. 

Use Cases

1) Brand Governance - Where organization wants to have control over which document or information should go to brand guidelines, With this APIs you can automate your item workflows and call these APIs when someone approves the items etc. or in general an automation can be placed to as and when brand guideline updates quarterly or annually, these automation can keep updating brand kits as per updated guidelines.

2) Marketing Automation - Where a company with global brand and regional brands would like to set up tone of voice or campaign specific to region and they would like automate the process of updating the section or sub section of brand kits with regional tone or localize campaigns.

3) External Integration - When scenarios like you would like to read a tone of voice of the brand and use that tone of voice to send email with that tone or generate and email from some system by passing tone of voice so brand feels the same, at that time these APIs can also be used.

4) Ease of brand management - It becomes difficult for a company with so many brand to keep updating the brand specific document or sections as and when brand refreshes or rebranding happens, in those scenarios automation will help sync brands and keep updating them with latest and greatest.

I will be sharing some example and code snippets on GITHUB of above use cases in next blogs

Thanks for reading

Sitecore Stream - Configure AI capabilities into Sitecore XP using Stream Platform DxP package

Hi Folks,

Recently, I delivered a webinar on "Do more in era of less with AI capabilities, Ideation to execution with Sitecore Stream" and I also have a blog around the same which you can read here on Sitecore Stream - Do more in era of less with AI capabilities, Ideation to execution

The purpose of the webinar session and above blog is to first set the tone for reader and audience to understand what Sitecore Stream is and how it operates and what values it can bring in, if possible, I will share the slides too so you can go through it.

That session and the blog above were more from how Sitecore Stream is solving marketer's challenges

Today, I will be showing you how you can configure the power of stream and its Brand Aware AI capabilities right into your on-prem XP, Why XP is because all other SaaS based (Cloud based) DxP products like XMC, Search, CH already has the power of Stream, So I will show you the steps of configuring Stream into on-prem XP version.

Prerequisites: 

To be able to configure stream in your XP, Following are the prerequisites 

1) You will need Sitecore Stream license, Your on-prem XP is actually going to talk to your SaaS Stream and its configuration of brand-kit etc to generate content, so you need Stream license.

2) Your Sitecore version should be >=10.2 to be able to use this.

3) Sitecore Stream for Platform DxP package or module, which you will install using package installation wizard.

Configuration Steps

Step-1: First of all, to download the package you will need to login to https://portal.sitecorecloud.io/ 

Step-2: If you have stream subscription then and have access to Stream App, you will see that on the launchpad, so click on the Stream App

Step-3: Click on "Sitecore Stream for Platform DXP" and it will download a package

Step-4: Open your on-prem Sitecore instance and install the package you downloaded and verify the files to ensure that installation was successful.

NOTE: It is important to note that, when you download the package, it ships with other compressed folders too, so you can not give the downloaded package directly to Sitecore, but make sure to unzip it and only use folder called "SitecorePackage" to be uploaded in installation wizard of Sitecore package installation.

Step-5: To verify if the package installation was successful You can go to dashboard and see Stream section

So now, you have all binaries and config files now available which can talk to the SaaS Stream app in your subscription, now it is time to configure the connection to the Stream app so we can start getting the AI capabilities built into Sitecore.

Step-6: Configuring the connection to Sitecore Stream involves adding connection string of stream in connectionstring.config file, Connection string consists of "Client_id, client_secret, authority, audience"

While authority and audience values are fixed and global to the installation (authority=https://auth.sitecorecloud.io and audience=https://api.sitecorecloud.io), you just need to generate client_id and secret from Sitecore Stream portal itself from the "Admin" menu as shown in below screenshot. (Make sure you note them down somewhere, else they are only displayed once).

Step-7: Now write that values into connection string and put it in the our standard sitecore's connectionstring.config file

HINT: What i did was the package ships with all the helpful documentation, there is a "docs" folder in the package which has all the "MarkDown - .md files, which also helps with step by step processes, I referred to install.md and i updated my client_id and client_secret into the sample given in install.md and copied the whole connectionstring from there.

<add

name="sitecore.aiclient.stream"

connectionString="client_id=YOUR_CLIENT_ID;client_secret=YOUR_CLIENT_SECRET;authority=https://auth.sitecorecloud.io;audience=https://api.sitecorecloud.io"

/>

Step-8: How to make your local XP brand aware? You have configured the stream connection, but you would want to generate the content in context of your brand, right? if you do not know what is "Brand aware -AI" in Stream, please refer to my blog post on Stream where I have explained this.

Again, you will find how to configure brand-kit connection into your XP, is given in install.md file which ships with the package, here are the steps

1. Get your Brand Kit ID from the Stream Portal URL

2. Get your Organization ID from the Sitecore Portal URL

3. Update the following settings in `Sitecore.AiClient.config` located at "\App_Config\Sitecore\AiClient" folder

 <setting name="Sitecore.AiClient.Stream.BrandKitId" value="<<BRAND_KIT_ID>>" />

 <setting name="Sitecore.AiClient.Stream.BrandKitReferencePath" value="/api/brands/v1/organizations/org_<<ORG_ID>>/brandkits/<<BRAND_KIT_ID>>" />

You can get the brand kit id by going to Stream app -> Brand Kits -> Select your brand kit 

and click on your brand it and find the brand kit id in URL address bar of the browser

NOTE: Above brand kit configurations is for platform DxP package 1.0, In versions later than this brand kit selection can be done directly from Stream dialog box.

That is it my friends :) You connected and configured Sitecore Stream into your XP, now to check that, go to content editor and you will see AI icon

Content generation is possible with following field types

1) Single-line text

2) Multiline text

3) Rich text

Step-9: Let's open content editor and check AI icons on above field types

Step-10: See how you can use generative AI right into the editor and power of AI using Stream, Click the AI icon on the "Content" field and it will open Stream window, See I have requested Stream AI to generate the content

Thing to observe is, I have grounded the AI data using the brand-kit, so it knows how to output brand related content, for demo of my webinar I already grounded data for SUGNAGPUR and you can see AI is outputting it exactly that, that is called "Brand-Aware AI"

On top of it, you can also do things like "Make it shorter", "Improve writing", "Fix spelling and grammar", "Make longer", "Change Tone" (like soft, hard, friendly) using the button available on top of stream dialog box.

Read more on what are the possible option here

Summary

As you can the power of AI and its capabilities can be configured right into your Sitecore XM/XP (>-10.2 versions) and that also having brand awareness

This is pretty neat and need of the hour, which will make content authors and marketer's life easy and efficient.

Sitecore Stream - Do more in era of less with AI capabilities, Ideation to execution

Hi Folks,

This blog post is around my point of view from technologist on AI capabilities and orchestration provided by Sitecore Stream, and why it is important to know what it has to offer and RoI for a business and what it has for future roadmap, Because every customer has similar problems and as a technologist you should always try to solve those business problems, and hence below statement comes to my mind  for Sitecore Stream.

A true innovation should start from "Looking at the business problem and then choosing the tech to solve, and not vice versa"

With Sitecore Stream , This is exactly what Sitecore team has done, The product team and technical team worked with marketers and marketing team and with customer to understand their problems and then came up with this amazing innovation and AI capabilities throughout their composable stack.

I will try to simplify things for end audience, so it is easy for them to understand

What is Sitecore Stream? A new Product?

Before we even talk about real life challenges which Sitecore Stream solves, Let's try understanding in extremally simply manner what it Stream.

"Well, It is not a new DxP product, But it is the AI capabilities already available in all the Sitecore DxP products, It is designed to make life of marketers and content authors easy by giving them their brand specific content, and information on tip of their hand with chat like interface."

As we talk, These AI capabilities are already there across all products built in, The biggest win is content authors and marketers get AI capabilities not only while creating marketing campaign but right into their content authoring environment too with XP, XM, XMC what not, With use of GEN AI and power of Microsoft Azure Open AI, Now they can generate the content right from the editor itself.

Now Let's understand what the problems were and why something like this was needed, how was life of a marketers before it and after it.

Problems of marketers or content creators in every organization

Well, generally marketers follow the workflow, where to get the campaign out or getting a page live, they need to go through typical workflow

Now consider following challenges

1) Multiple brands and multiple channels.

2) Lot of old and historic assets and documents on different platforms.

3) Comply with brand guides of all the brands, not all documents are handy and takes lot of time.

4) Make sure they are aligning with the brand strategy, going through all of those documents to align with the brand strategy.

5) Finally make sure tone of voice, fonts, visual guidelines, Grammar align with the brand.

Lot of work right, and in today's world for marketers "There is less budget, less time" but still they need to get "Faster go to market", this is a real challenge and Sitecore Stream comes for a rescue for the very same reason

On top of above challenge, BetterBreifs research says 35% of marketing budget is wasted due to poor briefs, now consider AI at your help, more productive and as they say "Smarter - Strategic - Secure".

How Sitecore Stream Helps in Above Scenario?

1) Brand Aware AI

Sitecore Stream grounds the AI data model to learn against the brand documents uploaded and injected into the portal, This is very SMART way of making sure that all the AI capabilities around the products are targeted specific to your brand, So it is aware about your brand guidelines like tone of voice, which colors to use, which fonts, which images and what not

When you search anything on stream portal or ask anything to AI, It will now give or generate the content or campaign or briefs using your own brand context, which is a huge win.

2) Copilots & Agents

In Stream, there are copilots and agents which helps enhancing productivity by working along with you and doing automated tasks so that you can focus on actually doing what you are supposed to do, either strategy on your next campaign or focus on content which is the at the center and focus of Sitecore

There are multiple types of copilots which are responsible for different things

Brand Copilot

It makes sure that the generated content is aligned with brand's tone, color, voice and identity and context, It gets this context from brand kit documents uploaded from brand-kit section.

Content Copilot

Using this copilot content is generated from the brand kit context, It make sures and assist content authors or markets to add or edit content within the editor.

 

NOTE: Above screenshot is from one of the POC I am doing, I am excited to share it in my next blog

Experience Copilot

It facilitates the feedback loop kind of experience where it helps in content generation, image generation and then gives marketers to edit that content as per their need and run that cycle till content is satisfactory, majorly used for content hub to perform natural language search, visual search, tags images etc. 

3) Agentic Workflows 

It covers overall ideation to execution; End-to-end workflow automation and content creation life cycle using agents and continues cycle of generating a content till the validation and getting final output.

NOTE: Well, One might think that increasing use of the AI might take marketers or content author's job as now it knows the brand and context and also it completely automates lot of things using GEN AI using these copilots.

But the fact is, it will actually increase the productivity of marketers to think about strategy, and they will do more of those work which they should actually be doing and let the technology do these tasks.

Sitecore Stream portal is the orchestration layer, so it's like central collaborative manner task where you can project manage you campaign get the things done 

Technical Insights

Well, Let's talk something about what powers Stream and technology behind it.

1) Sitecore Stream uses Microsoft Azure OpenAI Services, that gives access to LLMs (Large Learning Models) and RAG (Retrieval Augmented Generation) which are grounded and trained on brand specific data and documents, it currently uses GTP-4 model, which gives amazing "Brand Aware" AI power.

Some says "Sitecore + Microsoft AI" - Match made in haven :)

2) You can use AI capabilities of Sitecore Stream on any DxP platforms, be it XMC, Search, Content Hub, XP, XM

For XP or XM, you can start using Stream from Sitecore 10.2 onwards and upwards. (Depending on the license and other things but Sitecore Stream supports >=10.2 versions)

See below table to understand which product has what feature of stream already available as you read it (This might change depending on future roadmap or some changes might be there and already implemented too)

(Image Source Credit: From the webinar "Supercharge your marketing engine with AI" by Kelly Masters,Sr. Product Manager at Sitecore and Michele Fisher, Global Director of Business Strategy at Microsoft" Find Here)

3) Sitecore Stream can be accessed in two ways 

• Using online stream app through the Sitecore Cloud Portal

        Login to Stream Cloud Portal and find the Sitecore Stream App (I am going to have a detailed demo and blog around it soon of how stream app is used.)

• From within your composable stack and Sitecore products.

        AI Capabilities within the products are powered by Stream and it is already built into them, and you can start using it today.

4) With capabilities in DxP, you can generate content using AI for your specific brand as well as you can generate multiple versions of the content too.

5) You can do content translations using Strem AI capabilities.

What is the ROI? and why a customer should use Sitecore Stream?

This is my favorite part and section because the whole purpose of this blog so that it adds value to customers using Sitecore and thinking about getting ROI.

Before 4-5 years we all know if a marketing team or content team needs to get the campaign out or create a content for specific brand of a customer, they had to do briefing and find right asset also scan through historical asset and campaigns and find the right style and font which matches with their brand

Before 2-3 years we had external APIs for generative AI using which we could create content, also we could create campaign using ChatGPT and time to market was made little faster.

But today with Stream, Marketers/Content team gets their brand specific asset, font, style, documents everything as per their brand and they literally get it by just asking on using the same chat as prompt and it will generate content right for your brand which is Complient, Proprietary and secure.

One of the case study or a use case is the brand "Nestle", They partnered with Sitecore on the journey to solve those issues of marketers using AI capabilities using Stream. Watch Here

Nestle has over 2000 brands and its huge, consider just to get a campaign out, how many brand strategic document one will need to go through and make sure it aligns with the policy and brand before it is actually live, that is where Stream gave them ROI and faster go to market.

NOTE: If you have a question like we can do the same in ChatGPT or Gemini and get the job done, but remember one thing, It is like having a general knowledge of all kind, it does not know your brand and content it produces could not be compliant for your brand, Where with Sitecore Stream Brand Aware AI, it super charges your content creation and ideation or execution. 

Future Roadmap

Well future of Sitecore Stream is extremally promising, remember these points are subject to change as per the roadmap and plan from Sitecore.

1) Component Creation using AI

There will be possibility of component creation without writing any code, this will be a game changer where marketing team will not have dependency on a developer to create a component, but it will be possible to output the components using NLPs - Natural Language Processing.

2) Creating Content and Variants

More improvements in the way content is being generated using GEN-AI and making it useful of content authors and marketers to generate variations of content and images.

There are so many other AI capabilities planned but I only have listed which are shared publicly on the web or in Sitecore webinars. 

Summary

“Content is a king and still a king, but without relevance it's nothing”, when we say relevance it's again the experiences, so with relevant content, it captures attention, it drives more engagement and of course it creates a strong brand loyalty.

And there for you know having stream is needed in this digital space is because you need things faster, relevant, faster go to market and with top notch experiences.

NOTE: With Sitecore Stream customer data is fully secure.

Next blog post, I will show the actual business use case and how it addresses the faster go to market and so much of productivity by actually demoing the product.

Sitecore 10.2 Headless SXA is looking for "master" db on CD server

Hi Folks,

Sometimes you are trying to troubleshoot one thing and end up solving another :), How many of you have ended up same? 

Well, I was actually troubleshooting the high CPU and no. of requests on CD was trying to find some error patterns from Sitecore logs for one of the issues we were facing, there was one specific error which we kept observing was below

Exception

and if you drill further down into it, Sitecore was looking for master DB on CD servers, which is not possible, so definitely, this is bug

I previously faced the same error but for other assembly and to troubleshoot and resolving that the approaches were different but for both meaning of the error is same, for 10.3 where i got the error is drafted here Error in Sitecore 10.3 XM with CDs - Could not find configuration node: database/database[@id='master']

Troubleshooting

It was clear that because some the code which is in "VersionInitializer" class of "SItecore.XA.Foundation.Upgrade.dll" is referring to "master" database on a CD server.

So, I just tried dissembling the DLL to understand what's going on and you can see below

It is clearly seen that code is referring to wrong DB instead of the context DB, it is "master" DB and that is the reason of the error

Solution

Further troubleshooting and research suggested that this can be safely ignored, and this comes only when the instance is starting, but just to keep our logs clean we just commented out below lines from "\App_Config\Modules\SXA\Foundation\Sitecore.XA.Foundation.Upgrade.config" of CD instance

and that is the suggested approach from Sitecore.

Sitecore XM Cloud Components - BYOC - Bring in your existing react or nextjs component library

Hi All,

Last year i did lot of research and hands on to "XM Cloud Component Builder", Very powerful futuristic tooling provided by Sitecore.

At that time it was still in "Early Access" phase.

You can have a look at my "SUGCON 2023 - Presentation on XM Cloud Component Builder"  to know about it's capabilities and use cases.

and if you are PODCAST lover, find the podcast here 

I would highly recommend going through above video or podcast to understand "Low code, No code" and what is "FeaaS" and power of it, As LCAPs (Low Code Application Platforms) are growing at the speed of around 25% every year, And it's a future.

Now, Let's talk about Part-2 of it, more advance use cases of "XM Cloud FeaaS", Once you have understanding of the component builder, you can do wonders, and this part-2 we will talk about something amazing it can do which is BYOC, Bringing Your Own Components.

What is BYOC - Bring Your Own Code (Or Components)

In Part-1, this feature BYOC was still under development, but now it is live from long time and available and as its name suggests, you can technically bring your already developed react / NextJS component directly into XM Cloud, Developers can bring their own code (BYOC) by referencing an external code base that includes components so that they can be used in the component builder interface.

Sounds so powerful right? Also, it could be great sellable point or a feature where, you do not have to rewrite your business code which you had in those components which you have created in react or NextJS, that means you can almost get them working straight into XM Cloud.

It opens endless possibilities for those customers who already have their component libraries and the sites using those components, and with composable features of XM Cloud, they can reuse their code too.

My Experience working with it.

 

There are three options I have personally configured and work like a charm.

1) If you have XM Cloud Starter Kit Repo and XM Cloud Project already, it has seamless support where you can just do certain configurations and your react/NextJS components which are in XM Cloud StarterKit code will come to life in component builder, we will see the steps later in this blog

2)  If you have existing NextJS components already, you can bring those components straight to your builder & start using them on "Pages".

3) If you have existing react components already, you can bring those components straight to your builder & start using them on "Pages".

You can bifurcate these approaches in two broader categories.

1) Using an existing rendering host of XM Cloud - When your components are made part of XM Cloud Startkit which you are already using, in that case you do not require to add any external rendering host, XM Cloud default rendering host will take care of it.

2) Using "External Rendering Host" - In cases where your react/NextJS components are already residing in some other source control and live on some other sites and which you want to bring into XM Cloud component builder, Here it needs the live URL of the site to be configured into XM Cloud, If the components are configured correctly (which I will show you in this blog), XM Cloud will read those components too.

What are the steps?

Ok, Let's talk about the steps first, 

1) If you are trying to use your react or NextJS components to be made available, then first you will need to install below in your code repo

npm install @sitecore-feaas/clientside

2) Go to your component which you want to import and write below line 

/* eg. src/components/MyComponent.js */

import * as FEAAS from "@sitecore-feaas/clientside/react"

3) You will need to register the component

FEAAS.registerComponent(ExampleClientsideComponent, {

  name: "clientside-only",

  title: "Clientside-only component",

  description: "Description of my example component",

  thumbnail:

    "https://feaasstatic.blob.core.windows.net/assets/thumbnails/byoc.svg",

  group: "D Section",

  required: ["firstName"],

  properties: {

    firstName: {

      type: "string",

      title: "First name",

    },

    lastName: {

      type: "string",

      title: "Last name",

    },

    telephone: {

      type: "number",

      title: "Telephone",

      minLength: 10,

    },

    bold: {

      type: "boolean",

      title: "Show text in bold weight",

    },

  },

  ui: {

    firstName: {

      "ui:autofocus": true,

      "ui:emptyValue": "",

      "ui:placeholder": "Write your first name",

    },

    bold: {

      "ui:widget": "radio",

    },

  },

});

Remember BYOC support is out of the box for XM Clodud Starter repo, above steps are only for if you want your existing NextJS component to be registered and available in XMC as external rendering host

I think it is very powerful way of working where you can just Bring Your Code in and it just works, That means you do not have to rewrite those components for xm cloud but you will be able to see those components exactly like your current react site and will be able to drag & drop them on the pages

Reference - Walkthrough: Registering an external React component | Sitecore Documentation