SharePoint Online Backup in Azure DocumentDB Tool – First step with DocumentDB

Hello all !

Microsoft has announced a crazy new service in Azure, the no SQL Database DocumentDB. DocumentDB can store objects in JSON (in flat files). What you need to know : to manage the database, you can use C#, JavaScript Server-Side (new Microsoft concept, which strength to step back ^^) and any other language that support REST API. DocumentDB Queries are very similar to SQL Queries (no CAML or other unforgettable SharePoint stuff), so you won’t be lost.

If you want more than this too really short presentation, please read this article.

There are a lot of pro and cons against those new kind of database (No SQL). So, my curiosity was picked. What can we do with those new DocumentDB service ? What are limitations, crazy new stuff, …

I think it would be a good exercise to discover Azure DocumentDB by developing a new tool. As you maybe know, I’m crazy about Azure, but also about SharePoint. So, I decided to experiment Azure DocumentDB by developing a tool to backup your SharePoint Online Sites, Lists, Libraries in DocumentDB.

This development is not for use in production (please don’t). If, one day, the tool is strong enough, I will open a CodePlex project. But we are sooooo far from that. In this example, I only backed up List Items (with no support of specific fields types). Nothing more.

What do we need to develop a good SharePoint Online Backup Tool in Azure DocumentDB ?

  • Visual Studio 2015 Preview (or Visual Studio 2013 with the last update and Microsoft Azure DocumentDB Client Library NuGet Package)
  • A SharePoint Online Site Collection
  • An Azure account to use :
    • DocumentDB
    • WebJob (don’t forget it’s a cloud backup tool, you’ll need to schedule some tasks)

Another last thing before coding : I did that in three hours during my week-end. It’s not optimized, not beautiful, not really « object-oriented ». This is only a first approach of DocumentBD, and it will be  improved in a next blog post. Trust me.

We spend to munch time in explanations. LET’S ROCK WITH AZURE NOW ! In the new portal preview, of course.

1. Click on « New », at the bottom of the page.Create your Document DB

Feed all of that boring fields, and click on « Create ». Note that the creation can take 10 minutes. It’s enough time to watch the SharePoint Blues Band (twice, because the video lasts 4:22).

2. After enjoying this cultural time, please write down (in NotePad, for example) the URI and Primary Key. You’ll need it later. Really.

To access those informations, please go to your Azure Dashboard, and click on the DocumentBD you’ve just created.

Azure DashBord

And then click on « Keys » (I really love this new interface, thank you MS. The old one was just… Old)

Keys

You can see the URI, and the Primary Key, called, in my foreign but beautiful language : URI and Clé Primaire.
Those keys are a way to access you database and execute operations on it. There are two « admin keys », and also two « Read Only keys ».
I think there is users management in this kind database, but I didn’t spend some time on right management. Let’s see it another time. You see that you can regenerate your keys by clicking on some buttons at the top of the window, if needed.

The database is ready. Let’s Rock With Visual Studio (and Azure) now !

3. Create a new Project, Application Console, or Azure WebJob (almost the same thing).
Usings you’ll need :

using Microsoft.Azure.Documents;
using Microsoft.Azure.Documents.Client;
using Microsoft.Azure.Documents.Linq;
using Microsoft.SharePoint.Client;
using Newtonsoft.Json;
using System;
using System.Collections.Generic;
using System.Security;
using System.Linq;
using System.Text;
using System.Threading.Tasks;

4. SharePoint Online Connection, and Datas Retrieving.

Prepare a specific class to store datas of your SharePoint List. As I wrote above, I only code the backup for a SharePoint List. In the Progam.cs, add this :

 public class BackupedList
    {
        public string ListName;
        public List<BackupedItem> Items = new List<BackupedItem>();
        public class BackupedItem
        {
            public List<Values> ValuesList = new List<Values>();
            public class Values
            {
                public string Column;
                public string Value;
            }
        }
    }

Let’s connect to SharePoint :

using (ClientContext clientContext = new ClientContext("The URL of Your SharePoint"))
            {
                //Auth stuff
                SecureString ssPass = new SecureString();
                private static BackupedList blDeclarations = new BackupedList();
                foreach (char c in "PASSWORD".ToCharArray()) ssPass.AppendChar(c);
                clientContext.Credentials = new SharePointOnlineCredentials("YOUR O365 ACCOUNT", ssPass);
                //Get the list
                Web wCurrent = clientContext.Web;
                List lDeclarations = wCurrent.Lists.GetByTitle("THE TITLE OF YOUR LIST");
                //Get the items
                CamlQuery cqAllItems = new CamlQuery();
                ListItemCollection licItems = lDeclarations.GetItems(cqAllItems);
                //Load
                clientContext.Load(licItems);
                clientContext.ExecuteQuery();

                //Get datas
                foreach (ListItem liCurrent in licItems)
                {
                    BackupedList.BackupedItem biCurrent = new BackupedList.BackupedItem();
                    //Prepare datas
                    foreach (KeyValuePair<string, object> dicItemValues in liCurrent.FieldValues)
                    {
                        BackupedList.BackupedItem.Values vCurrent = new BackupedList.BackupedItem.Values();
                        vCurrent.Column = dicItemValues.Key;
                        if (dicItemValues.Value != null)
                        {
                            //TODO : other values types management 
                            vCurrent.Value = dicItemValues.Value.ToString();
                        }
                        biCurrent.ValuesList.Add(vCurrent);
                    }
                    blDeclarations.Items.Add(biCurrent);
                    //TODO : Attachments management
                }
            }
    }

We stored all of the list items values in blDeclarations.

5. DocumentDB !!!
Prepare the field by adding all of those methods to your code.
Many thanks to Ryan Crawcour for that. It was really helpful. 

        /// <summary>
        /// Get the database by name, or create a new one if one with the name provided doesn't exist.
        /// </summary>
        /// <param name="id">The name of the database to search for, or create.</param>
        private static async Task<Database> GetOrCreateDatabaseAsync(string id)
        {
            // Create a query object for database, filter by name.
            IEnumerable<Database> query = from db in client.CreateDatabaseQuery()
                                          where db.Id == id
                                          select db;
            // Run the query and get the database (there should be only one) or null if the query didn't return anything.
            // Note: this will run synchronously. If async exectution is preferred, use IDocumentServiceQuery<T>.ExecuteNextAsync.
            Database database = query.FirstOrDefault();
            if (database == null)
            {
                // Create the database.
                database = await client.CreateDatabaseAsync(new Database { Id = id });
            }
            return database;
        }

        /// <summary>
        /// Get a DocumentCollection by id, or create a new one if one with the id provided doesn't exist.
        /// </summary>
        /// <param name="dbLink">The Database SelfLink property where this DocumentCollection exists / will be created</param>
        /// <param name="id">The id of the DocumentCollection to search for, or create.</param>
        /// <returns>The matched, or created, DocumentCollection object</returns>
        private static async Task<DocumentCollection> GetOrCreateCollectionAsync(string dbLink, string id)
        {
            DocumentCollection collection = client.CreateDocumentCollectionQuery(dbLink).Where(c => c.Id == id).ToArray().FirstOrDefault();
            if (collection == null)
            {
                collection = await client.CreateDocumentCollectionAsync(dbLink, new DocumentCollection { Id = id });
            }

            return collection;
        }
    }

Then the code to launch the copy to the DocumentDB :

//DocumentDB
string databaseId = "spolbckpdb";
string collectionId = "listbckp";

 //Read the DocumentDB endpointUrl and authorisationKeys from config
 //These values are available from the Azure Management Portal on the DocumentDB Account Blade under "Keys"
 //NB > Keep these values in a safe & secure location. Together they provide Administrative access to your DocDB account
string endpointUrl = "THE URI YOU WROTE BEFORE";
string authorizationKey = "THE KEY !";
using (client = new DocumentClient(new Uri(endpointUrl), authorizationKey))
{
      try { RunDemoAsync(databaseId, collectionId).Wait(); }
      catch (Exception ex) { Console.WriteLine(ex.ToString()); Console.Read(); }
}

private static async Task RunDemoAsync(string databaseId, string collectionId)
{
     //Get, or Create, the Database
     var database = await GetOrCreateDatabaseAsync(databaseId);
     //Get, or Create, the Document Collection
     var collection = await GetOrCreateCollectionAsync(database.SelfLink, collectionId);
     //One approach to access to DocumentDB documents in .NET is through “POCOs” or plain-old-clr-objects
     //which will use JSON.NET serialization and deserialization under the covers to transmit the data over the wire
     Document created = await client.CreateDocumentAsync(collection.SelfLink, blDeclarations);
}

Please find the Visual Studio Project here.

You can debug, everything should be fine. If not, please tell me in comments.

Known issues :
Error during debug « The authorization token is not valid at the current time (…) ».
Solution :  https://social.msdn.microsoft.com/Forums/azure/en-US/1b75f98c-5b99-4224-8a20-8d90752019d5/the-authorization-token-is-not-valid-at-the-current-time?forum=AzureDocumentDB

6. Schedule the backup with Azure WebJobs
This is one of the best tools in Azure, it’s WebJobs. In Visual Studio, right click on your project, and then « Publish as Azure WebJob » and schedule the task !

Azure WebJob

I’d like to thanks those guys :
https://code.msdn.microsoft.com/Azure-DocumentDB-NET-Code-6b3da8af#content
http://azure.microsoft.com/en-us/documentation/articles/documentdb-sql-query/
http://weblogs.asp.net/scottgu/azure-new-documentdb-nosql-service-new-search-service-new-sql-alwayson-vm-template-and-more
http://www.vrdmn.com/2013/01/authenticating-net-client-object-model.html
http://stackoverflow.com/questions/20950628/authenticating-client-with-office-365-sharepoint-online

Next time, I will improve my ugly code, and of course, read in DocumentDB to add restoring features  !

Thank you for your time, and stay tuned !

Publicités

Laisser un commentaire

Entrez vos coordonnées ci-dessous ou cliquez sur une icône pour vous connecter:

Logo WordPress.com

Vous commentez à l'aide de votre compte WordPress.com. Déconnexion / Changer )

Image Twitter

Vous commentez à l'aide de votre compte Twitter. Déconnexion / Changer )

Photo Facebook

Vous commentez à l'aide de votre compte Facebook. Déconnexion / Changer )

Photo Google+

Vous commentez à l'aide de votre compte Google+. Déconnexion / Changer )

Connexion à %s