Sitecore – Cooking with Sitecore

DEF – Pipeline batch ribbon button issue

This is going to be a short blog for fixing issue I came across while working on custom pipeline creation and execution for Sitecore’s Data Exchange Framework(DEF or DxF).

The issue

While the Pipeline Batch item is selected, buttons on DATA EXCHANGE toolbar at top does not behave how they should.

Immediately after selecting pipeline item, I noticed all buttons in ribbon is visible and clickable. Secondly, clicking on Run Pipeline Batch Button does not works. I was neither starting execution of pipeline nor displaying any visible error messages. See below image for more details:

The fix

I started investigation and collecting events what could have done the damage. Found that before last deployment it was behaving the way it should be. Huuummm… that’s good hint, isn’t it? The last release just deployed files to CMS server. This is soothing to my ears as investment scope is drastically reduced to files. The content automatically gone out of equation.

To fix this, I tried install DEF SDK package first with just deploying file artifacts and skipping any content installation.

Unfortunately, that does not work ….!!! 😦

Next best bet is obvious and try to re-install the DEF framework package with deploying files only.

And that does the trick ….!!!! (Dance)

Happy Data Sync and import using Sitecore Data Exchange Framework. 🙂

Sitecore Publishing Service – Part 3 – Advance Configurations

So far in Part – 1 and Part – 2 of this series, the Sitecore Publishing Service up & running in Azure PaaS and the Sitecore Publishing module is installed & configured.

This last part will trying to address few additional scenarios with publishing service. Scenarios like, the Sitecore databases are hosted in Azure SQL and Sitecore implementation have multiple publishing targets.

Sitecore database in Azure(Azure SQL)

If publishing service hosted on Azure web app and the databases are also in Azure SQL Database, there are few additional configurations needed for the service to work seamlessly with bigger publishing payload.

Follow below steps to configure publishing service for Azure Environment

Navigate to \configurations\system.webServer\aspNetCore node in wwwRoot\web.config file of publishing service in app service.
replace the arguments attribute with the following

arguments=" --environment azure"

Save the web.config
Stop and start app service

By default Sitecore publishing service is running under the “production” environment and that patches the configurations for production. After above change, service will start with Azure environment and it will patch azure environment specific configurations.

Navigate to wwwroot/config/azure
rename file sc.publishing.sqlazure.connections.xml.example to sc.publishing.sqlazure.connections.xml. Remove .example extension and make sure it has .xml extension.

New publishing target…!!!???

If other than default Internet publishing target is being used in CMS, there are few additional configurations needs to be made. So lets have a look at those.

I am assuming the additional publishing target also linked to additional database other than default web database. If so, the new connection string also needs to be added to publishing service.

New connection string

In this scenario, will take example of new web database in Azure SQL US region and need to add new target to publish content to web database to this US region.

Add new connection string to Publishing service. lets name it web_us

Sitecore.Framework.Publishing.Host configuration setconnectionstring web_us "<US web database connection string>"

/config/sitecore/publishing/sc.publishing.xml

Open above file and follow below steps:

navigate to Settings\Sitecore\Publishing\Services\DefaultConnectionFactory\Options\Connections
Search for the child node Internet, duplicate the whole node along with children. For example, name it InternetUS
Within that InternetUS node navigate to child node ConnectionString and change the value of the node to following to match the name of the connection sting in steps above.

${Sitecore:Publishing:ConnectionStrings:web_us}

Navigate to Settings\Sitecore\Publishing\Services\StoreFactory\Options\Stores\Targets
Search for the child node Internet, duplicate the whole node along with children. For example, name it InternetUS
Change the value of the ConnectionName to InternetUs. This must match the name of the name of the new node name under Settings\Sitecore\Publishing\Services\DefaultConnectionFactory\Options\Connections. Created in one of the step above.
Set the id of new Publishing Target item in Sitecore CMS. For example,

<Id>46FDC9AF-3F00-4D55-BDB8-86ED13F82E00</Id>

Set the value of child node ScDatabase to newly added databsae name. In our example it would be web_us.
Save the file

wwwroot/config/azure/sc.publishing.sqlazure.connections.xml

Note
This step is optional if the service is not using databases on Azure SQL.

Navigate to Settings\Sitecore\Publishing\Services\DefaultConnectionFactory\options\Connections\
Search for Internet node in child nodes and duplicate that node with name InternetUS along with child nodes
Save File

Bonus

Log level changing for debugging

Nothing can beat detailed level logging while debugging application/service. Default publishing service have two filters one sitecore and anther is default.

For sitecore filter the default log level is set to Information and default log level is set to Warning. See below xml for the same.

<Filters>
    <Sitecore>Information</Sitecore>
    <Default>Warning</Default>
</Filters>

One can select log filter level to one of the following:

Trace : The most detailed log messages, may contain sensitive application data. These messages are disabled by default and should never be enabled in a production environment.
Debug : Logs that are used for interactive investigation during development should use the Debug level. These logs should primarily contain information useful for debugging and have no long-term value.
Information : Track the general flow of the application using logs at the Information level. These logs should have value in the long term.
Warning : Warnings should highlight an abnormal or unexpected event in the application flow. This event does not cause the application execution to stop, but can signify sub-optimal performance or a potential problem for the future.
Error : An error should be logged when the current flow of execution is stopped due to a failure. These should indicate a failure in the current activity, not an application-wide failure. These will mainly be unhandled exceptions and recoverable failures.
Critical : A critical log should describe an unrecoverable application or system crash, or a catastrophic failure that requires immediate attention.

To overwrite these default log level filters, use wwwroot/config/sitecore/sc.logging.xml file.

Note
The Development environment mode sets the Sitecore log level filter to Debug. So the service can be run under Development Environment for more detailed logging.

After all these configuration changes restart the publishing service.

Have a happy performing Sitecore publishing…!!!!

Sitecore Publishing Service – Part 1 – Install and Configure Service in Azure PaaS

While working on finding solution for slow publishing issue of more than 10,000 items in Azure PaaS, I came across this very powerful add-on Sitecore Publishing Services.

Believe it or not but once it is installed, configured and up, it reduced the publishing time drastically. In our case, publishing of those 10,000 items were taking around 1 hour 30 mins(hhhuuuffff…), merely took 12 mins to publish.(Super duper awesome, isn’t it…??? This makes everyone’s life easy)

Thus, I decided to write blog on whole solution. Blog will become very lengthy if I put everything in one. Thus decided to split in in three parts. Part 1 will cover installation and basic configuration of publishing service. Part 2 will cover installation of publishing module in CMS. The third part will cover advance configuration of publishing service.

So, lets get started…

Download

You can download both Sitecore publishing service and publishing module from Sitecore’s development site here. It is the designed and developed by Sitecore itself.

Note:
Just check the version compatibility with Sitecore version where you wanted to implement this publishing service/module. see image below for more details. (As I have downloaded first the latest and greatest version 4 at the time I am writing this blog for Sitecore 9.0.1 and later figure out it is not compatible.)

Install publishing service on Azure PaaS

With my current client all their Sitecore implementation is in Azure PaaS. It absolutely makes sense to have this Sitecore Publishing service also setting in Azure PaaS. So, follow below steps to get the publishing services up as app service.

Download the service application as zip file from above link. Select the one which is matching your host architecture type either 32 bit or 64 bit.
Open Kudu – Advance tool for the web app where you want to host publishing service

in Kudu editor open command console by navigate to Debug Console menu option and select CMD
navigate to site root folder using following command

cd site\wwwroot

Drag and drop service zip file to folder view area above console. This will upload the service package to Azure web app.
Once upload completed, run following command on console below the folder view

unzip -o *.zip

Assumption
The site’s wwwroot folder has only one zip file which is recently uploaded publishing service zip file.

Once unzipping of application completed, execute following commands in CMD. This will set connection stings of core, master and web database in publishing service. Going forward, these connection strings will be used for all publishing jobs.

Sitecore.Framework.Publishing.Host configuration setconnectionstring core "<core database connection string>"

Sitecore.Framework.Publishing.Host configuration setconnectionstring master "<master database connection string>"

Sitecore.Framework.Publishing.Host configuration setconnectionstring web "<web database connection string>"

Once connection strings are configured for these databases the next step is to upgrade the database schema to required one for Sitecore publishing service

Sitecore.Framework.Publishing.Host schema upgrade --force

After executing above command the console displays the result of the command. The below screenshot have result with already updates schema for all the databases as I have previously upgraded the schema.

Once above commands are successfully executed, now it is the time to check the status of the publishing service web application.

In the browser open below URL

http://<<azure app service name>>.azurewebsites.net/api/publishing/maintenance/status

Above web api will return following JSON result if all the configurations are correctly set.

{
    "status": 0
}

This are basic steps to deploy Sitecore Publishing services up and running in azure PaaS. In the Part – 2, will cover installation of Sitecore Publishing Module in Sitecore CM/CA with configuration.

Content sync with RAZL script

Hi Sitecore Enthusiastic,

Recently come across very great strategy to sync content between two instances of Sitecore using a very powerful tool called Razl form Hedgehog.

Please read more about Razl here.

I have used this tool before mainly to compare the content between two different databases. For example production CM and Pre-production CM. Today, I came across very new and great features with latest version of Razl offering call tasks and executing task using Razl scripts.

It is offering content sync tasks to be exported in a file(or call script) as xml format and later one can use that to sync data in different environment using Razal script.

For more information about Razl task, read official document here.

The great part of this task is you can cherry pick what you want to sync out of the whole Sitecore content tree. You can go deep up to the field level with your sync task.

once you have a list of tasks ready, you simple can export that as script and execute it with command or create scheduler task to run that script using command and it will do the sync between source and target connections.

Razl.exe /script:Migrate.xml
Razl.exe /script:"c:\Site Migration\Razl.xml"

Even if you want to go one level further you can automate this sync task post publishing(perhaps not ideal, if frequent publishing is happening) or you can hook it post deploy task in you CD script(if your build contains TDS/unicorn content items).

You must be wondering this is great tool and great way to sync, but what if I am trying to sync big chunk of content items!!!???.
You can use another great feature of Razl called Lightening Mode(obviously you can configure this in you script as well). This mode really boost the performance of sync script if your CoplyAll task is configured to do bulk copying of items.

As far as logging concern, Razl Script mode will output actions as they happen to the command prompt, this output can be written to a text file if required using following command sample.

Razl.exe /script:"c:\Site Migration\Razl.xml" >> c:\logs\RazlScriptLogs.txt

Tip: Single arrow > will create the file if missing and if exist, overwrite the content of file.
Double arrow >> will create the file if missing and if exist, it will append to the file.

If a log folder has been already configured in the Razl GUI than Script mode will also write to the log file under configured log folder.

Thanks for reading.

Happy Content Sync…!!!

Reference
Script – https://hedgehogdevelopment.github.io/razl/script.html
Connections – https://hedgehogdevelopment.github.io/razl/connections.html
Lightning mode – https://hedgehogdevelopment.github.io/razl/comparing.html#lightning-mode

Set dynamic datasource for template field in Sitecore multi-site

This is to share some of the tricks you can use while dealing with multi-sites with Sitecore CMS while designing content solutions.

While dealing with multi-site, we all come across scenario where we want data of some fields to be different per site based on where that item exist. In other words, want to load dynamic data in link types of fields for the data present in that local site.

Of course, the answer is use Sitecore query instead of fixed path. The trick I wanted to share is how to formulate that query.

Lets take one example(I feel if you explain with real life example, people are tend to understand more quickly).

My multi-site content information structure is laid out as follow:

Content

SiteA

Home (All the navigable pages goes here)

Configuration (Master settings for site and features)

Data Library (As name suggested this holds data for the site)

SiteB

Home

Configuration

Data Library

Now, lets assume you have SiteA for Asia-Pacific and SiteB for Europe and there are country data items present under Data Library for both respected sites as shown in picture below.

multi-site-information-structure

Now, you want content editor to select only countries present under that site in field called “Supported Countries” for site’s Configuration item.

sitea-configuration-for-country-items-as-datasource

You can have following Sitecore query as Source of field Supported Countries for the Site Configuration template.

query:./ancestor::*[@@templateid='{B8313310-198B-4596-9532-7E39BAB33503}']/Data Library/*[@@templateid='{7CF67E0D-A30F-4C48-9247-69DE07821520}']/*

Let’s dive in details about how this query is formulated. Before doing that, please thoroughly go through image below.

sc-query-explained

The query is divided in the six parts:

Part 1: first part is query keyword, which informs Sitecore that the data source of this field is not just simple item path but complex Sitecore query.
Part 2: The the .(dot) present the current item. These items are highlighted in Red. In our example, it is Configuration item. As we are setting query for the field Supported Countries of template Site Configuration. Configuration item under SiteA and SiteB is actual instance of template.
Part 3: This is the tricky part, form current item we are getting the list of all the ancestor items. For Configuration item under SiteA, the ancestors are SiteA, Content and Sitecore. For SiteB, ancestors to Configuration items are SiteB, Content and Sitecore. These items are highlighted in violet color. Once we get the list of ancestors, we need to filter out the item which is of template type Site(id of this template is {B8313310-198B-4596-9532-7E39BAB33503}.
Part 4: From Filtered Site item, we need to dive in the the sub-tree of Library. To do this we can use the same method of filtering based on template id like we did in part 3 above, but this would be bit costly operation as we need to look for all children underneath site and then filter based on Site Data Library template. Here, I have used fixed path instead, assuming all the sites in this content tree have to have a item called “Data Library”. This is to demonstrate, one can use fixed path query as well with Sitecore query. These are highlighted in green color in content tree.
Part 5: The 5th part is same as part 3, except looking for ancestors, it is looking for children and then filtering the child with the template id {7CF67E0D-A30F-4C48-9247-69DE07821520}. This is to find folder type item of template Country Folder, which are highlighted in Sky-blue color in content tree.
Part 6: The last part is to get the list of all items under the Countries folder filtered in part 5 above. Here, I assuming none other item present under this folder except for template type Country. If you are looking for particulate type of items you can use filters to get those items. These items in content tree are highlighted in grey color as leaf nodes.

Tip: Be cautious while using descendant or //* in field’s datasource query. This is costlier read operations and may hit the performance of field rendering in Content editor.

Enjoy dynamic field data source and Sitecore query …!!!

Reference
https://sdn.sitecore.net/Reference/Using%20Sitecore%20Query/Sitecore%20Query%20Syntax.aspx

Ongoing publishing issue with Sitecore 7.2

Here I wanted to share simple solution to an ongoing issue of publishing with Sitecore 7.2 with my company. It is such a simple solution but took lots of efforts and time to get to it. Thought to share it so that if someone is facing same kind of issue, can be benefited out of it.

The problem:

Out marketing team(who is also responsible for the content management) continuously reporting us with the issue of publishing. They mentioned that whenever we try to publish any item, the publishing took lot of time to finish. Most of the time publishing dialog stuck at Initialization stage. This issue sometimes go worst, even after waiting for half an hour to 45 minutes the publishing dialog still shows initialization and we need to ask web admin guys to re-cycle application pool and start publishing again(this is worst case scenario).

So as a first step, I asked them to avoid bulky publishing. I also suggested then to avoid using publishing related items rather go to individual items changed and only publish those items. The reason is to minimize items count in publishing which keeps publishing process as short as possible and keep them moving.

Even though content authors follow my suggestions, the issue still persist. So we started investigating even more. We planned to take help of Sitecore support personal who visiting us once every month. He suggested us to compare all the config files with pure vanilla version and along with all the other files and folder so we can be absolute sure we are having all the goodies what should be there. After this huge activity we identified couple of missing files and configurations in both CM and CD environments.

So we have gathered all those missing pieces and deployed to the respected environments.

After this we have reported a side effect from marketing, that what ever we are deploying to Fail-over publishing target(which is our DR cum internal Preview servers) it also eventually published to production(which is very strange to us and this was not the behavior before we deploy the missing bits).

Root cause

So, I started investigating in context to new side effects. While going through the log file for CM server, my eyes struck with an strange log entry which looks something like below.

5768 2016-03-10 11:03:57 INFO AUDIT (ad\paragd): Execute workflow command. Item: master:/sitecore/content/Site1/Home/test, language: en, version: 11, id: {23005A37-6BB5-4025-AC5B-047A8D3F3719}, command: /sitecore/system/Workflows/Site1 Workflow/Draft/Submit, previous state: Draft, next state: /sitecore/system/Workflows/Site1 Workflow/Awaiting Approval, user: ad\paragd
ManagedPoolThread #3 2016-03-10 11:03:59 INFO Starting update of index for the database 'master' (1 pending).
ManagedPoolThread #3 2016-03-10 11:03:59 INFO Update of index for the database 'master' done.
ManagedPoolThread #16 2016-03-10 11:03:59 INFO Starting update of index for the database 'master' (1 pending).
ManagedPoolThread #16 2016-03-10 11:03:59 INFO Update of index for the database 'master' done.
5284 2016-03-10 11:04:00 INFO AUDIT (ad\paragd): Execute workflow command. Item: master:/sitecore/content/Site1/Home/test, language: en, version: 11, id: {23005A37-6BB5-4025-AC5B-047A8D3F3719}, command: /sitecore/system/Workflows/Site1 Workflow/Awaiting Approval/Approve, previous state: Awaiting Approval, next state: /sitecore/system/Workflows/Site1 Workflow/Approved, user: ad\paragd
5284 2016-03-10 11:04:00 INFO AUDIT (ad\paragd): [Publishing]: Starting to process 2 publishing options
4572 2016-03-10 11:04:00 INFO HttpModule is being initialized
568 2016-03-10 11:04:00 INFO HttpModule is being initialized
5456 2016-03-10 11:04:01 INFO Job started: Publish
2736 2016-03-10 11:04:01 INFO Job started: Publish to 'web'
.
.
.
2736 2016-03-10 11:04:01 INFO Job ended: Publish to 'web' (units processed: 11)
7052 2016-03-10 11:04:01 INFO Job started: Publish to 'web_Failover'
.
.
.
7052 2016-03-10 11:04:02 INFO Job ended: Publish to 'web_Failover' (units processed: 11)

Above lines, I found in the log just above the log line for manually triggering of publishing just for the single publishing target which is “fail-Over”.

4392 2016-03-10 11:04:18 INFO AUDIT (ad\paragd): Publish item: master:/sitecore/content/Site1/Home/test, language: en, version: 11, id: {23005A37-6BB5-4025-AC5B-047A8D3F3719}
3600 2016-03-10 11:04:28 INFO AUDIT (ad\paragd): Publish, root: {23005A37-6BB5-4025-AC5B-047A8D3F3719}, languages:en, targets:WWW_Failover, databases:web_Failover, incremental:false, smart:true, republish:false, children:false, related:false
3600 2016-03-10 11:04:28 INFO AUDIT (ad\paragd): [Publishing]: Starting to process 1 publishing options
2864 2016-03-10 11:04:29 INFO Job started: Publish
5240 2016-03-10 11:04:29 INFO Job started: Publish to 'web_Failover'

Solution

From above behavior on the log file, it was clear that the workflow has to do something with this. When I checked the default workflow assigned, I found that there is an auto publishing action configured with final stage of the workflow as shown in below screen grab.

But alone having auto publish action configured, should not be making publishing this slow. So I have investigated more and found the real culprit.

The auto published action is configured with deep=1 as parameters.
So now, I have dive into the de-compiled code and found this is very dangerous parameter configured with auto publishing action.
Having deep parameter set, you are instructing auto publisher to publish everything underneath the selected item(approving item here) to every publishing targets(in this case 2 targets).

So, if you are approving leaf node/item, the auto publishing will be quick as it will only publish that node along with no child. But if you are approving, lets say home node of the site, you are actually publishing not just that root home item but pretty much the whole site to all available publishing targets…!!!(Very shocking…isn’t it?).

So now after approving the item, when you trying to publish that home node to selected target, it always queue up in the publishing queue and Sitecore publishing dialog box show you Initialization message screen as shown above. The item publishing will be picked up when previous auto publishing job is done. This is the case with just one editor, if you have 5-6 content editor, then the situation can go into really worst.

So the easy solution is to remove that auto publishing action item which was publishing lot of unwanted items to all targets in background without any visible signs.

So we did that and everyone is happy ever after…..

But the story does not ended here. I was thinking while going to home that day, when the auto publishing action was configured and present with default workflow since the edges then why approving the item wasn’t picked by both the CD servers?

So next day, I have kept my investigation going and discover the ScalabilitySettings.config file which was missing on all CD servers. While deploying the missing configurations and file we have deployed this file on all CD serves started picking the events execution for all CDs.

So, happy publishing….!!!!

Reference

https://sitecorebasics.wordpress.com/2011/05/28/is-your-sitecore-publishing-stucks/

Sitecore Active Directory Configuration: Read User Data from AD Group

The problem:

while working on the Allowing access to the organization’s users as Content Author on the Sitecore CM server using Sitecore’s Active Directory(AD) Module, we encounter very strange issue of not able to read user data from the AD user group. So the idea is to enable group of employees in organization to grant access to the CMS server to do the content authoring duties and enable their windows credential to get access to the Sitecore CMS.

In other words Sitecore’s User Manager wasn’t displaying user from active directory group.

Steps to reproduce issue/problem

Install Sitecore’s Active Directory Module from here.

Configure AD connection string to point to a logical group in the active directory structure.

<connectionStrings>
    <add name="ManagersConnString" connectionString="LDAP://testsrv/OU=Managers,DC=testdomain,DC=sitecore,DC=net" />
</connectionStrings>

Configuring the ASP.NET Security Providers(Membership Provider)

 <add name="ad"
 type="LightLDAP.SitecoreADMembershipProvider"
 connectionStringName="AD"
 applicationName="sitecore"
 minRequiredPasswordLength="1"
 minRequiredNonalphanumericCharacters="0"
 requiresQuestionAndAnswer="false"
 requiresUniqueEmail="false"
 connectionUsername="[put the username here]"
 connectionPassword="[put the password here]"
 connectionProtection="Secure"
 attributeMapUsername="sAMAccountName"
 enableSearchMethods="true" />

Activating Switching Providers

In web.config file, in <system.web> section, browse for <membership> element and find the provider called sitecore and set its realProviderName attribute to switcher.

<membership defaultProvider="sitecore" hashAlgorithmType="SHA1">
    <providers>
        <clear/>
        <add name="sitecore" type="Sitecore.Security.SitecoreMembershipProvider, Sitecore.Kernel" realProviderName="<strong>switcher</strong>" providerWildcard="%" raiseEvents="true"/>
        <add name="sql" type="System.Web.Security.SqlMembershipProvider" connectionStringName="core" applicationName="sitecore" minRequiredPasswordLength="1" minRequiredNonalphanumericCharacters="0" requiresQuestionAndAnswer="false" requiresUniqueEmail="false" maxInvalidPasswordAttempts="256"/>
        <add name="switcher" type="Sitecore.Security.SwitchingMembershipProvider, Sitecore.Kernel" applicationName="sitecore" mappings="switchingProviders/membership"/>
    </providers>
</membership>

Open Sitecore’s User Manager Check the user listed from AD.

Here we are just interesting in getting the users from the Active Directory not the Roles. But you can extent Sitecore Securities to get the roles and the some of the additional User related data using configuring Role Manager and Profile Provider settings.

Root Cause

There are no actual user data present under the AD group. we have checked that, if you change the connection string to the node in the AD tree which contains the actual user under it will work. But if you are trying to point your AD connection to a Logical Group it won’t work.

Solution :
While dealing with the AD most of the time we have great difficulty to see and check what’s the structure and values of element/user properties.
For that I personally recommend the simple ans easy to use tool called “AD Explorer”. It is free and handy tool which gives you the graphical representation of you AD tree and also helps you to find the correct connection string.

It’s free tool and you can download it from https://technet.microsoft.com/en-us/library/bb963907.aspx

Step 1

Change the connection string to point to the parent node of the logical group.

<connectionStrings>
    <add name="ManagersConnString" connectionString="LDAP://testsrv/DC=testdomain,DC=sitecore,DC=net" />
</connectionStrings>

Step 2

In the system.web/membership/providers add the attribute customFilter for the “AD” node as shown in below configuration done in step 2 in reproduction steps above.

<add name="ad"
type="LightLDAP.SitecoreADMembershipProvider"
connectionStringName="AD"
applicationName="sitecore"
minRequiredPasswordLength="1"
minRequiredNonalphanumericCharacters="0"
requiresQuestionAndAnswer="false"
requiresUniqueEmail="false"
connectionUsername="[put the username here]"
connectionPassword="[put the password here]"
connectionProtection="Secure"
attributeMapUsername="sAMAccountName"
enableSearchMethods="true"
customFilter="(memberOf=CN=Managers,DC=testdomain,DC=sitecore,DC=net)" />

Once it is all done, Open Sitecore’s User Manager and check there are users listed from AD. It would be looks like image below.

Happy Active Directory Configuration 🙂