Parallelize ProcessAdd with AMO

In this article we see a function to enable parallelism in commands sent to Analysis Services by using the AMO library even when you need to use the change the QueryBinding. This function is useful because the current AMO library (up to SQL Server 2012) cannot correctly generate the required XMLA code if multiple commands using QueryBinding are executed within the same batch.

When you want to implement an incremental process using ProcessAdd, chances are that you want to change the QueryBinding within the ProcessAdd command, because you put directly in the SQL code the WHERE condition in order to extract only new rows from the data source.
If you use AMO to send Process commands to Analysis Services, you already have all you need at your hands. In fact, the following code can be used to add rows to an existing partition.

using AMO = Microsoft.AnalysisServices;

AMO.Server server = ... // Initialize the server
AMO.DataSourceView dataSourceView = ... // Get the data source view
AMO.Partition partition1 = ... // Get the partition
string query1 = "SELECT * FROM table1 WHERE newRow = 1";

partition1.Process(
    AMO.ProcessType.ProcessAdd,
    new AMO.QueryBinding(
            dataSourceView.DataSourceID,
            query1
    )
);

In the previous query you run a ProcessAdd on a single partition and the command executes immediately. If you want to process more partition, you might want to execute both operations in parallel. In Tabular this is actually possible when the two partitions does not belong to the same table. In order to execute two or more process commands in the same transaction in parallel, you have to set to true the CaptureXml property of the server connection: in this way, all of the XMLA commands are saved in an internal collection (CaptureLog) and a single batch can be sent to the server by calling the ExecuteCaptureLog method of the AMO server object, like in the following code.

using AMO = Microsoft.AnalysisServices;

AMO.Server server = ... // Initialize the server
AMO.DataSourceView dataSourceView = ... // Get the data source view
AMO.Partition partition1 = ... // Get the partition
string query1 = "SELECT * FROM table1 WHERE newRow = 1";
AMO.Partition partition2 = ... // Get the partition
string query2 = "SELECT * FROM table2 WHERE newRow = 1";

server.CaptureXml = true;
partition1.Process(
    AMO.ProcessType.ProcessAdd,
    new AMO.QueryBinding(
            dataSourceView.DataSourceID,
            query1
    )
);
partition2.Process(
    AMO.ProcessType.ProcessAdd,
    new AMO.QueryBinding(
            Database.DataSourceView.DataSourceID,
            query2
    )
);
server.ExecuteCaptureLog(true, true);

Unfortunately, the code above does not work because a QueryBinding was specified in the two Process commands. This is the error generated:

The syntax for the 'Process' command is incorrect. The 'Bindings' keyword cannot appear under a 'Process' command if the 'Process' command is a part of a 'Batch' command and there are more than one 'Process' commands in the 'Batch' or the 'Batch' command contains any out of line related information. In this case, the 'Bindings' keyword should be a part of the 'Batch' command only.


The error description is very clear. We cannot rely on the existing feature that generates the batch command because it has an XMLA syntax that is not valid. In order to workaround the problem, I wrote a C# function that extends the AMO.Server class, read the XMLA code generated by the existing library and moves the QueryBinding nodes out of the Process nodes, putting them directly in the Batch command. The goal is that we will replace this line in the previous example:

server.ExecuteCaptureLog(true, true);

with the following one:

server.ExecuteCaptureLogWithBindings(true, true);

The following code is the implementation of the ExecuteCaptureLogWithBindings. I tested it in .NET 4.0.

using AMO = Microsoft.AnalysisServices;

static class Tools {
    /// <summary>
    /// Execute the captured commands from AMO Server
    /// </summary>
    /// <param name="server">Server connection that captured AMO commands</param>
    /// <param name="transactional">true if the batch must be executed in a single transaction</param>
    /// <param name="parallel">true if commands have to be executed in parallel</param>
    /// <returns></returns>
    public static AMO.XmlaResultCollection ExecuteCaptureLogWithBindings(this AMO.Server server, bool transactional, bool parallel) {
        AMO.XmlaResultCollection results = null;
        bool captureXml = server.CaptureXml;
        try {
            if (!server.Connected) {
                throw new InvalidOperationException("Server is not connected");
            }

            XNamespace ns = "http://schemas.microsoft.com/analysisservices/2003/engine";
            var commands = 
                from string command in server.CaptureLog
                select XElement.Parse( command );

            XElement batch =
                new XElement(
                    ns + "Batch",
                    new XAttribute( "Transaction", transactional ),
                    new XElement( 
                        ns + "Bindings",
                        from processCommand in commands
                        select processCommand.Elements(ns + "Bindings").Elements(ns + "Binding").First()
                    ),
                    (parallel) ? 
                        // In case it is parallel, all commands are within Parallel
                        new XElement[] { new XElement(
                            ns + "Parallel",
                            from cmd in commands
                            select new XElement(
                                cmd.Name,
                                from node in cmd.Elements() 
                                where node.Name != ns + "Bindings" 
                                select node
                            )
                        ) }
                    :
                        // If it is not parallel, all commands are within the Batch 
                        from cmd in commands
                            select new XElement(
                                cmd.Name,
                                from node in cmd.Elements() 
                                where node.Name != ns + "Bindings" 
                                select node
                            )
                );
            Console.WriteLine(batch);
            results = server.Execute( batch.ToString() );
        }
        finally {
            server.CaptureXml = captureXml;
        }
        return results;
    }
}

By using the ExecuteCaptureLogWithBindings function you will be able to assign the QueryBinding property to single Process commands in both Tabular and Multidimensional models.