Sitecore CMS not showing graph on Launch pad


This morning when I login to one of none production environment I found that the graph was not loading on Sitecore CMS launch pad.

When I did into more details and found out the ajax request to get the data are failing with server error(HTTP-500). As shown in below image the response of the API call does not providing any information or clue.

Immediately, I jumped onto logs and found following entry in there.

 The remote server returned an error: (403) Forbidden.

The full exception looks like below.

50616 09:48:50 ERROR [Experience Analytics]: System.Net.WebException: The remote server returned an error: (403) Forbidden.
   at System.Net.HttpWebRequest.GetResponse()
   at Sitecore.Xdb.Reporting.Datasources.Remote.RemoteReportDataSourceProxy.GetData(ReportDataQuery query)
   at Sitecore.Xdb.Reporting.ReportDataProvider.c__DisplayClass18_0.b__1()
   at Sitecore.Xdb.Reporting.ReportDataProvider.GetCachedDataForQuery(ReportDataQuery query, Func`1 ifNotAvailableInCache, Nullable`1 ifModifiedSince)
   at Sitecore.Xdb.Reporting.ReportDataProvider.ExecuteQueryWithCache(ReportDataQuery query, ReportDataSource dataSource, CachingPolicy cachingPolicy)
   at Sitecore.Xdb.Reporting.ReportDataProvider.GetData(String dataSourceName, ReportDataQuery query, CachingPolicy cachingPolicy)
   at Sitecore.ExperienceAnalytics.Api.ReportDataService.ExecuteQuery(IReportQueryData queryData, CachingPolicy cachingPolicy)
   at Sitecore.ExperienceAnalytics.Api.ReportingService.RunQuery(ReportQuery reportQuery)
   at Sitecore.ExperienceAnalytics.Api.Http.AnalyticsDataController.Get(ReportQuery reportQuery)

Fix

After further investigation found out the Reporting keys were not aligned between CM and Reporting server in the ConnectionStrings.config file.

<add name="reporting.apikey" connectionString="84090dc97055b10532a054da067df368" />

Further reading

Razl Service Timeout Issue


While using Razl syncing couple of large content sub-trees under Sitecore this morning, it has prompted for below error after progressing for while.

Error 'The HTTP request to 'http://xxx.xxx/_CMP/RazlService.asmx' has exceeded the allotted timeout of 00:04:59.9990000. The time allotted to this operation may have been a portion of a longer timeout.' getting Sitecore items.

The fix for above issue is to increase timeout period for the http request to classic web service. After applying following fix, it has started working as expected.

  • Navigate to the root folder of the website mentioned in above error message
  • navigate to _CMP folder and open web.config file for editing
  • Add following line to the web.config file under <system.web> node
<httpRuntime targetFramework="4.6.2" maxRequestLength="512000" executionTimeout="600" />
  • Save the web.config file
  • Close all running instances of Razl
  • Re-run Razl

Happy Content Sync, folks…!!!

Sitecore Publishing Service – Part 3 – Advance Configurations


So far in Part – 1 and Part – 2 of this series, the Sitecore Publishing Service up & running in Azure PaaS and the Sitecore Publishing module is installed & configured.

This last part will trying to address few additional scenarios with publishing service. Scenarios like, the Sitecore databases are hosted in Azure SQL and Sitecore implementation have multiple publishing targets.

Sitecore database in Azure(Azure SQL)

If publishing service hosted on Azure web app and the databases are also in Azure SQL Database, there are few additional configurations needed for the service to work seamlessly with bigger publishing payload.

Follow below steps to configure publishing service for Azure Environment

  • Navigate to \configurations\system.webServer\aspNetCore node in wwwRoot\web.config file of publishing service in app service.
  • replace the arguments attribute with the following
arguments=" --environment azure"
  • Save the web.config
  • Stop and start app service

By default Sitecore publishing service is running under the “production” environment and that patches the configurations for production. After above change, service will start with Azure environment and it will patch azure environment specific configurations.

  • Navigate to wwwroot/config/azure
  • rename file sc.publishing.sqlazure.connections.xml.example to sc.publishing.sqlazure.connections.xml. Remove .example extension and make sure it has .xml extension.

New publishing target…!!!???

If other than default Internet publishing target is being used in CMS, there are few additional configurations needs to be made. So lets have a look at those.

I am assuming the additional publishing target also linked to additional database other than default web database. If so, the new connection string also needs to be added to publishing service.

New connection string

In this scenario, will take example of new web database in Azure SQL US region and need to add new target to publish content to web database to this US region.

  • Add new connection string to Publishing service. lets name it web_us
Sitecore.Framework.Publishing.Host configuration setconnectionstring web_us "<US web database connection string>"

/config/sitecore/publishing/sc.publishing.xml

Open above file and follow below steps:

  • navigate to Settings\Sitecore\Publishing\Services\DefaultConnectionFactory\Options\Connections
  • Search for the child node Internet, duplicate the whole node along with children. For example, name it InternetUS
  • Within that InternetUS node navigate to child node ConnectionString and change the value of the node to following to match the name of the connection sting in steps above.
${Sitecore:Publishing:ConnectionStrings:web_us}
  • Navigate to Settings\Sitecore\Publishing\Services\StoreFactory\Options\Stores\Targets
  • Search for the child node Internet, duplicate the whole node along with children. For example, name it InternetUS
  • Change the value of the ConnectionName to InternetUs. This must match the name of the name of the new node name under Settings\Sitecore\Publishing\Services\DefaultConnectionFactory\Options\Connections. Created in one of the step above.
  • Set the id of new Publishing Target item in Sitecore CMS. For example, 
<Id>46FDC9AF-3F00-4D55-BDB8-86ED13F82E00</Id>
  • Set the value of child node ScDatabase to newly added databsae name. In our example it would be web_us.
  • Save the file

wwwroot/config/azure/sc.publishing.sqlazure.connections.xml

Note

This step is optional if the service is not using databases on Azure SQL.
  • Navigate to Settings\Sitecore\Publishing\Services\DefaultConnectionFactory\options\Connections\
  • Search for Internet node in child nodes and duplicate that node with name InternetUS along with child nodes
  • Save File

Bonus

Log level changing for debugging

Nothing can beat detailed level logging while debugging application/service. Default publishing service have two filters one sitecore and anther is default.

For sitecore filter the default log level is set to Information and default log level is set to Warning. See below xml for the same.

<Filters>
    <Sitecore>Information</Sitecore>
    <Default>Warning</Default>
</Filters>

One can select log filter level to one of the following:

  • Trace : The most detailed log messages, may contain sensitive application data. These messages are disabled by default and should never be enabled in a production environment.
  • Debug : Logs that are used for interactive investigation during development should use the Debug level. These logs should primarily contain information useful for debugging and have no long-term value.
  • Information : Track the general flow of the application using logs at the Information level. These logs should have value in the long term.
  • Warning : Warnings should highlight an abnormal or unexpected event in the application flow. This event does not cause the application execution to stop, but can signify sub-optimal performance or a potential problem for the future.
  • Error : An error should be logged when the current flow of execution is stopped due to a failure. These should indicate a failure in the current activity, not an application-wide failure. These will mainly be unhandled exceptions and recoverable failures.
  • Critical : A critical log should describe an unrecoverable application or system crash, or a catastrophic failure that requires immediate attention.

To overwrite these default log level filters, use wwwroot/config/sitecore/sc.logging.xml file.

Note

The Development environment mode sets the Sitecore log level filter to Debug. So the service can be run under Development Environment for more detailed logging.

After all these configuration changes restart the publishing service.

Have a happy performing Sitecore publishing…!!!!

Sitecore Publishing Service – Part 2 – Install and Configure Module


Now that Publishing service is deployed, configure and up in Part – 1, the next bit is to install and configure Sitecore publishing service module in Sitecore CM and CD. This part is very simple and strait forward. So lets get started…

The Sitecore publishing module is nothing but Sitecore package and needs to be installed the same way as other Sitecore packages. This package contains all the necessary artifacts for this module to perform it’s job.

Download

The Sitecore Publishing Module can be downloaded from here.

Note

Just check the the version compatibility with Sitecore version where you wanted to implement this publishing service/module. For this you need to navigate to individual release page and check the compatibility detail at start of the page under the main heading.

Install publishing module

To install Sitecore publishing module, one can use OOTB Sitecore installation wizard.

==========================================================
***Important! Copy and save this information***
==========================================================
    BEFORE YOU CLICK NEXT:
    - Ensure you have installed and configured the Sitecore Publishing Service (this module only enables integration with the service)
        Documentation detailing how to install the service is available seperately.

        [Warning] This module will not work without a properly configured service instance.  No items will be able to be published.
    
    AFTER YOU CLOSE THE WIZARD:
    After the package is installed, follow these steps to complete the Sitecore Publishing Module installation:
    - Configure the service endpoints:
        Add a configuration file which overrides the 'PublishingServiceUrlRoot' setting to point to your service module        
        Make sure the address contains a trailing slash
        e.g.
            <?xml version="1.0" encoding="utf-8"?>
            <configuration xmlns:patch="http://www.sitecore.net/xmlconfig/">
              <sitecore>
                <settings>
                  <setting name="PublishingServiceUrlRoot">http://sitecore.publishing/</setting>
                </settings>    
              </sitecore>
            </configuration>
    - Configure the Content Delivery Servers:
        Ensure that the following file is in the website 'App_Config/Modules/PublishingService' directory:
            * Sitecore.Publishing.Service.Delivery.config
        Ensure that the following files exist in the website 'bin' directory:
            * Sitecore.Publishing.Service.dll
            * Sitecore.Publishing.Service.Abstractions.dll
            * Sitecore.Publishing.Service.Delivery.dll

Note

Just before installation process starts, installer will prompt the information about configurations and artifacts needed for the Sitecore Module and services to work. As instructed, copy and save that instruction shown in above(From version 3.1.3).

Configure

Once installation of the module is successful, in CM perform following steps:

For CM

For example:

http://my-sitecore-publishing-service.azurewebsites.net/

Note

Make sure the URL is post-fix with forward slash(/) at the end.

For CD

  • Navigate to <website root>/App_Config/Modules/PublishingService folder in both the CM and CD app services
  • Copy Sitecore.Publishing.Service.Delivery.config file from CM to above folder in CD
  • Navigate to <website root>/bin folder in both the CM and CD app services
  • Copy below library files to CD’s bin folder from CM
    • Sitecore.Publishing.Service.dll
    • Sitecore.Publishing.Service.Abstractions.dll
    • Sitecore.Publishing.Service.Delivery.dll

At this point, the publishing will start working, if there is only one default Internet publishing target is there. The Publishing Module has overwritten few of Sitecore default settings for publishing. So after installation of publishing module, one may notice change in UI for the Publishing. Please read here for more details about visual changes and changes in types of publishing.

The last part -3 will cover some specific configuration for Azure PaaS environment and some advance setting of Sitecore Publishing Service. So stay tuned and enjoy content publishing….!!!

Sitecore Publishing Service – Part 1 – Install and Configure Service in Azure PaaS


While working on finding solution for slow publishing issue of more than 10,000 items in Azure PaaS, I came across this very powerful add-on Sitecore Publishing Services.

Believe it or not but once it is installed, configured and up, it reduced the publishing time drastically. In our case, publishing of those 10,000 items were taking around 1 hour 30 mins(hhhuuuffff…), merely took 12 mins to publish.(Super duper awesome, isn’t it…??? This makes everyone’s life easy)

Thus, I decided to write blog on whole solution. Blog will become very lengthy if I put everything in one. Thus decided to split in in three parts. Part 1 will cover installation and basic configuration of publishing service. Part 2 will cover installation of publishing module in CMS. The third part will cover advance configuration of publishing service.

So, lets get started…

Download

You can download both Sitecore publishing service and publishing module from Sitecore’s development site here. It is the designed and developed by Sitecore itself.

Note:

Just check the version compatibility with Sitecore version where you wanted to implement this publishing service/module. see image below for more details. (As I have downloaded first the latest and greatest version 4 at the time I am writing this blog for Sitecore 9.0.1 and later figure out it is not compatible.)

Install publishing service on Azure PaaS

With my current client all their Sitecore implementation is in Azure PaaS. It absolutely makes sense to have this Sitecore Publishing service also setting in Azure PaaS. So, follow below steps to get the publishing services up as app service.

  • Download the service application as zip file from above link. Select the one which is matching your host architecture type either 32 bit or 64 bit.
  • Open Kudu – Advance tool for the web app where you want to host publishing service
  • in Kudu editor open command console by navigate to Debug Console menu option and select CMD
  • navigate to site root folder using following command 
cd site\wwwroot
  • Drag and drop service zip file to folder view area above console. This will upload the service package to Azure web app.
  • Once upload completed, run following command on console below the folder view
unzip -o *.zip

Assumption

The site’s wwwroot folder has only one zip file which is recently uploaded publishing service zip file.
  • Once unzipping of application completed, execute following commands in CMD. This will set connection stings of core, master and web database in publishing service. Going forward, these connection strings will be used for all publishing jobs.
Sitecore.Framework.Publishing.Host configuration setconnectionstring core "<core database connection string>"
Sitecore.Framework.Publishing.Host configuration setconnectionstring master "<master database connection string>"
Sitecore.Framework.Publishing.Host configuration setconnectionstring web "<web database connection string>"
  • Once connection strings are configured for these databases the next step is to upgrade the database schema to required one for Sitecore publishing service
Sitecore.Framework.Publishing.Host schema upgrade --force

After executing above command the console displays the result of the command. The below screenshot have result with already updates schema for all the databases as I have previously upgraded the schema.

Once above commands are successfully executed, now it is the time to check the status of the publishing service web application.

In the browser open below URL

http://<<azure app service name>>.azurewebsites.net/api/publishing/maintenance/status

Above web api will return following JSON result if all the configurations are correctly set.

{
    "status": 0
}

This are basic steps to deploy Sitecore Publishing services up and running in azure PaaS. In the Part – 2, will cover installation of Sitecore Publishing Module in Sitecore CM/CA with configuration.

Sitecore config patching overridden for Azure…!!!???


Hi Folks,

Recently, I been struck by an issue where my config patches were not reflecting in Test environment. Not to mention, it is working absolutely fine in my local as well as on CI(typical developer’s excuse).

After initial investigation found, it is patching all of the config changes from my custom config patch file but not the particular one where I was overriding the Sitecore’s OOTB logging functionality to log error and fatal to Azure Application Insight(know as AppInsight).

For our deploying artifacts to test environment(which is Azure PAAS), we are using another great tool by Sitecore called Sitecore Azure Module(currently known as Sitecore Azure Toolkit).This is the one which is compatible with Sitecore 8.1. With this in place, my whole focus is shifted to this tools functionality and internal processing.

After thorough investigation of this tool and it’s architecture, I found the root cause of the config not patching issue(or I say config patching overriding issue). The module’s configuration item of template type /sitecore/templates/Azure/Deployment/Azure Deployment has a field call Global WebConfig Patch which has title text as [Do not edit!] – Global Web Config Patch (These are the global web config patches that will always be applied to every deployment). This field holds all the configurations for Sitecore XP to work in Azure PAAS environment.

Sitecore Azure Module Deployment Configuration Item

While we trigger the command to Upgrade Files in Sitecore Azure Module Window, it start packaging all the required files and also configurations. As part of Sitecore configuration packaging, it gets the whole patched output of all config files and after that is applies the config changes mentioned in the Global WebConfig Patch field of the above mentioned template under your Environment/location/Farm/WebRole item of your module configuration. The whole config path is finally written in single Sitecore.config file.

Tip: Be cautious while changing this global setting of Sitecore Azure module. Don’t do it until and unless you are knowing what are you doing.

Happy Deployment…!!!! 🙂

Content sync with RAZL script


Hi Sitecore Enthusiastic,

Recently come across very great strategy to sync content between two instances of Sitecore using a very powerful tool called Razl form Hedgehog.

Please read more about Razl here.

I have used this tool before mainly to compare the content between two different databases. For example production CM and Pre-production CM. Today, I came across very new and great features with latest version of Razl offering call tasks and executing task using Razl scripts.

It is offering content sync tasks to be exported in a file(or call script) as xml format and later one can use that to sync data in different environment using Razal script.

For more information about Razl task, read official document here.

The great part of this task is you can cherry pick what you want to sync out of the whole Sitecore content tree. You can go deep up to the field level with your sync task.

once you have a list of tasks ready, you simple can export that as script and execute it with command or create scheduler task to run that script using command and it will do the sync between source and target connections.

Razl.exe /script:Migrate.xml
Razl.exe /script:"c:\Site Migration\Razl.xml"

Even if you want to go one level further you can automate this sync task post publishing(perhaps not ideal, if frequent publishing is happening) or you can hook it post deploy task in you CD script(if your build contains TDS/unicorn content items).

You must be wondering this is great tool and great way to sync, but what if I am trying to sync big chunk of content items!!!???.
You can use another great feature of Razl called Lightening Mode(obviously you can configure this in you script as well). This mode really boost the performance of sync script if your CoplyAll task is configured to do bulk copying of items.

As far as logging concern, Razl Script mode will output actions as they happen to the command prompt, this output can be written to a text file if required using following command sample.

Razl.exe /script:"c:\Site Migration\Razl.xml" >> c:\logs\RazlScriptLogs.txt

Tip: Single arrow > will create the file if missing and if exist, overwrite the content of file.
Double  arrow >> will create the file if missing and if exist, it will append to the file.

If a log folder has been already configured in the Razl GUI than Script mode will also write to the log file under configured log folder.

Thanks for reading.

Happy Content Sync…!!!

Reference
Script – https://hedgehogdevelopment.github.io/razl/script.html
Connections – https://hedgehogdevelopment.github.io/razl/connections.html
Lightning mode – https://hedgehogdevelopment.github.io/razl/comparing.html#lightning-mode