Sorted By: Tag (mongodb)

Integration Testing with MongoDB with MongoDB.Testing Library

I have put together a library, MongoDB.Testing, which makes it easy to stand up a MongoDB server, create a random database and clean up the resources afterwards. Here is how you can start using it.
2015-12-05 21:06
Tugberk Ugurlu


Considering the applications we produce today (small, targeted, "micro" applications), I value integration tests way more than unit tests (along with acceptance tests). They provide much more realistic testing on your application with the only downside of being hard to pinpoint which part of your code is the problem when you have failures. I have been writing integration tests for the .NET based HTTP applications which use MongoDB as the data storage system on same parts and I pulled out a helper into library which makes it easy to stand up a MongoDB server, create a random database and clean up the resources afterwards. The library is called MongoDB.Testing and it’s on NuGet, GitHub. Usage is also pretty simple and there is also a a few samples I have put together.

Install the library into your testing project through NuGet:

Install-Package MongoDB.Testing -pre

Write a mongod.exe locator:

public class MongodExeLocator : IMongoExeLocator
{
    public string Locate()
    {
        return @"C:\Program Files\MongoDB\Server\3.0\bin\mongod.exe";
    }
}

Finally, integrate this into your tests:

[Test]
public async Task HasEnoughRating_Should_Throw_When_The_User_Is_Not_Found()
{
    using (MongoTestServer server = MongoTestServer.Start(27017, new MongodExeLocator()))
    {
        // ARRANGE
        var collection = server.Database.GetCollection<UserEntity>("users");
        var service = new MyCounterService(collection);
        await collection.InsertOneAsync(new UserEntity
        {
            Id = ObjectId.GenerateNewId().ToString(),
            Name = "foo",
            Rating = 23
        });

        // ACT, ASSERT
        Assert.Throws<InvalidOperationException>(
            () => service.HasEnoughRating(ObjectId.GenerateNewId().ToString()));
    }
}

That’s basically all. MongoTestServer.Start will do the following for you:

  • Start a mongod instance and expose it through the specified port.
  • Creates a randomly named MongoDB database on the started instance and exposes it through the MongoTestServer instance returned from MongoTestServer.Start method.
  • Cleans up the resources, kills the mongod.exe instance when the MongoTestServer instance is disposed.

If you are doing a similar sort of testing with MongoDB, give this a shot. I want to improve this based on the needs. So, make sure to file issues and send some lovely pull requests.

ASP.NET 5 Identity MongoDB Implementation

ASP.NET Identity will have a new version with ASP.NET 5 which is going to be version 3.0.0 and I gave it shot to implement ASP.NET Identity MongoDB data store.
2015-11-05 20:04
Tugberk Ugurlu


As with everything, ASP.NET Identity will have a new version with ASP.NET 5 which is going to be version 3.0.0. There are some changes on the interfaces but it’s not as drastic as others. By default, it provides Entity Framework implementation which I assume going to be compatible with any data storage system that can plug into Entity Framework (which is good). However, you need to provide a custom implementation if you want to support a data storage system which doesn’t support Entity Framework. MongoDB is one of them and I gave it shot to implement ASP.NET Identity MongoDB store. The result was really good:

mongodb-aspnet-identity

Library is available on NuGet as Dnx.Identity.MongoDB package and it supports beta8 runtime release. For now, it’s part of a sample project I am working on but it will probably make it into its own repository soon. I also have a sample here which is the fork of the original Identity sample. You can look at this commit to see what I had to do to make it support my custom provider which is not that bad, if you ignore the dependency injection dance I had to make. That’s because my implementation of ASP.NET Identity library doesn’t support IRoleStore. Don’t worry, you will not need this as you already have IUserClaimStore and also, there is an open issue to change the dependency injection hook a bit so that the IRoleStore would be optional.

Here are a few more details about this ASP.NET 5 Identity MongoDB implementation:

  • Currently, there is no documentation.
  • MongoUserStore is thread safe. You can register this as a singleton.
  • At the moment, there is no logging and nice exception handling for the implementation.
  • The implementation only supports dnx452 or above and it doesn’t support corefx as MongoDB .NET Client has no support for that.
  • The library doesn’t support SetUserNameAsync as the user name is also the Id of the user. So, you cannot change the Id.
  • The implementation requires you to pass an implementation of IMongoDatabase through the constructor and it persists data into a collection which is named "users". There is going to be a way to change the name of the collection soon in upcoming releases.
  • E-mail uniqueness is ensured through a unique MongoDB index. However, this will not function properly if you shard the users collection.
  • The implementation doesn’t persist changes unless you call one of the UpdateAsync, CreateAsync or DeleteAsync methods (which is what UserManager does).

Give it a try and you can file issues here for now and send pull requests. Enjoy :)

Securing MongoDB Access with Username and Password

My MongoDb journey continues :) and I had my first attempt to put a username and password protection against a MongoDB instance. It went OK besides some hiccups along the way :) Let's see what I did.
2014-05-01 10:44
Tugberk Ugurlu


My MongoDb journey continues :) and I had my first attempt to put a username and password protection against a MongoDB instance. It went OK besides some hiccups along the way :) Let's see what I did.

First, I downloaded the latest (v2.6.0) MongoDB binaries as zip file and unzipped them. I put all MongoDB related stuff inside the c:\mongo directory for my development environment on windows and the structure of my c:\mongo directory is a little different:

Screenshot 2014-04-30 13.50.16

In order to set up the username and password authentication, first I need to get the mongod instance up and running with the authorization on. I achieved that by configuring it with the config file. Here is how it looks like:

dbpath = c:\mongo\data\db
port = 27017
logpath = c:\mongo\data\logs\mongo.log
auth = true

With this config file in place, I can get the mongod instance up:

image

First, I need to connect to this mongod instance and create the admin user. As you can see inside my config file, the server requires authentication. However, there is a localhost exception if there is no user defined inside the system. So, I can connect to my instance anonymously (as I’m running on port 27017 on localhost, I don’t need to define anything while firing up the mongo shell):

image

All great! Let’s create the system user administrator. As everything else, this chore is nicely documented, too:

use admin
db.createUser(
  {
    user: "tugberk",
    pwd: "12345678",
    roles:
    [
      {
        role: "userAdminAnyDatabase",
        db: "admin"
      }
    ]
  }
)

We are pretty much done. We have a user to administer our server now. Let’s disconnect from the mongo shell and reconnect to our mongod instance with our credentials:

mongo --host localhost --port 27017 -u tugberk -p 12345678 --authenticationDatabase admin

image

I’m all there and I can see what my privileges at the server with this user are.

If you try to connect to this MonogDB server anonymously, you will see that you are still able to connect to it. This’s really bad, isn’t it? Not at the level that you think it would be at. The real story is that MongoDB still allows you to connect to, but you won’t be able to do anything as the anonymous access is fully disabled.

image

The bad thing here is that your server existence is exposed which is still an important issue. Just be aware of this fact before getting started.

The user we created still has restricted access to MonogDB server. If you want to have a user with unrestricted access, you can create a user with root role assigned. In our case here, I will assign myself the root role:

use admin
db.grantRolesToUser("tugberk", ["root"])

Resources

Order of Fields Matters on MongoDB Indexes

Order of Fields Matters on MongoDB Indexes. Let's see how with an example.
2014-04-12 18:52
Tugberk Ugurlu


As my MongoDB journey continues, I discover new stuff along the way and one of them is about indexes in MongoDB. Let me try to explain it with a sample.

First, create the below four documents inside our users collection:

db.users.save({ 
	_id: 1, 
	name: "tugberk1", 
	login: [
		{ProviderName: "twitter", ProviderKey: "232"}, 
		{ProviderName: "facebook", ProviderKey: "423"}
	]
});
	
db.users.save({ 
	_id: 2, 
	name: "tugberk23", 
	login: [
		{ProviderName: "twitter", ProviderKey: "3443"}
	]
});

db.users.save({ 
	_id: 3, 
	name: "tugberk4343", 
	login: [
		{ProviderName: "dropbox", ProviderKey: "445345"}
	]
});

db.users.save({ 
	_id: 4, 
	name: "tugberk98", 
	login: [
		{ProviderName: "dropbox", ProviderKey: "3443"}, 
		{ProviderName: "facebook", ProviderKey: "768"}
	]
});

Let’s query the users collection by login.ProviderKey and login.ProviderName:

db.users.find({
	"login.ProviderKey": "232", 
	"login.ProviderName": "twitter"
}).pretty();

image

It found the document we wanted. Let’s see how it performed:

db.users.find({
	"login.ProviderKey": "232", 
	"login.ProviderName": "twitter"
}).explain();

image

Result is actually pretty bad. It scanned all four documents to find the one that we wanted to get. Let’s put an index to ProviderName and ProviderKey fields:

db.users.ensureIndex({
	"login.ProviderName": 1, 
	"login.ProviderKey": 1
});

Now, let’s see how it performs the query:

image

It’s better as it scanned only two documents. However, we had only one matching document for our query. As the chances that the providerKey will be more unique than the ProviderName, I want it to first look for the ProviderKey. To do that, I need to change the index:

db.users.dropIndex({
	"login.ProviderName": 1, 
	"login.ProviderKey": 1 
});
db.users.ensureIndex({ 
	"login.ProviderKey": 1, 
	"login.ProviderName": 1 
});

Let’s now see how it’s trying to find the matching documents:

db.users.find({
	"login.ProviderKey": "232", 
	"login.ProviderName": "twitter"
}).explain();

image

Boom! Only one document was scanned. This shows us how it’s important to put the fields in right order for our queries.

Resources

A C# Developer's First Thoughts on MongoDB

After working with RavenDB over the year, I just started looking into MongoDB. I worked with MongoDB a year ago or so in a small project but my knowledge was mostly rusty and I don't want that to happen again :) So, here I'm, documenting what my second thoughts are :)
2014-04-12 14:22
Tugberk Ugurlu


After working with RavenDB over the year, I just started looking into MongoDB. I worked with MongoDB a year ago or so in a small project but my knowledge was mostly rusty and I don't want that to happen again :) So, here I'm, documenting what my second thoughts are :) The TL;DR is that: I'm loving it but the lack of transaction sometimes drifts on a vast dark sea. It's OK through. The advantages generally overcomes this disadvantage.

Loved the Mongo Shell

First thing I liked about MongoDB is its shell (Mongo Shell). 

3d6f5fdd4be53096b6992d5b84b0d7be

It makes it insanely easy for you to get used to MongoDB. After running the mongod.exe, I fired up a command prompt and navigated to mongo.exe directory and entered the mongo shell. Mongo Shell runs pure JavaScript code. That's right! Anything you know about JavaScript is completely valid inside the mongo shell. Let's see a few things that you can do with mongo shell.

You can list the databases on your server: show dbs

2

You can see which database you are connected to: db

3

You can switch to a different database: use <database name here>

4

You can see the collections inside the database you are on: show collections

5

You can save a document inside a collection: db.users.save({ _id: "tugberk", userName: "Tugberk" })

6

You can list the documents inside a collection: db.users.find().pretty()

7

You can run a for loop:

for(var i = 0; i < 10; i++) { 
	db.users.save({ 
		_id: "tugberk" + i.toString(), 
		userName: "Tugberk" + i.toString() 
	}) 
}

8

You can run the code inside a js file

9

saveCount.js contains the following code and it just gets the count of documents inside the users collection and logs it inside another collection:

(function() {
     var myDb = db.getSiblingDB('myDatabase'),
         usersCount = myDb.users.find().count();
         
     myDb.countLogs.save({
          count: usersCount,
          createdOn: new Date()
     });
}());

All of those and more can be done using the mongo shell. It's a full blown MongoDB client and probably the best one. I don't know if it's just me but big missing feature of RavenDB is this type of shell.

Loved the Updates

Update operations in MongoDB is just fabulous. You can construct many kinds of updates, the engine allows you to do this fairly easily. The one I found most useful is the increment updates. Increment updates allows you to increment a field and this operation will be performed concurrency in-mind:

db.books.update(
   { item: "Divine Comedy" },
   {
      $inc: { stock: 5 }
   }
)

The above query will update the stock filed by 5 safely.

Not much Love for the .NET Client

MongoDB has an official .NET client but MongoDB guys decided to call this "C# driver". This is so strange because it works with any other .NET languages as well. I have to say that MongoDB .NET client is not so great in my opinion. After coming from the RavenDB .NET client, using the MongoDB .NET client just feels uncomfortable (However, I’m most certainly sure that I’d love its Node.Js client as it would feel very natural).

First of all, it doesn't support asynchronous requests to MongoDB server. All TCP requests are being done synchronously. Also, there is no embedded server support. RavenDB has this and it makes testing a joy. Let's look at the below code which gets an instance of a database:

MongoClient client = new MongoClient("mongodb://localhost");
MongoServer server = client.GetServer();
MongoDatabase db = server.GetDatabase("mongodemo");

There is too much noise going on here. For example, what is GetServer method there? Instead, I would want to see something like below:

MongoClient client = new MongoClient("mongodb://localhost");
using(var session = client.OpenSession("myDatabase"))
{
     // work with the session here...
}

Looks familiar :) I bet it does! Other than the above issues, creating map/reduce jobs just feels weird as well because MongoDB supports JavaScript to perform map/reduce operations.

var map =
    "function() {" +
    "    for (var key in this) {" +
    "        emit(key, { count : 1 });" +
    "    }" +
    "}";

var reduce =
    "function(key, emits) {" +
    "    total = 0;" +
    "    for (var i in emits) {" +
    "        total += emits[i].count;" +
    "    }" +
    "    return { count : total };" +
    "}";

var mr = collection.MapReduce(map, reduce);
foreach (var document in mr.GetResults()) {
    Console.WriteLine(document.ToJson());
}

The above code is directly taken from the MongoDB documentation.

Explore Yourself

MongoDB has a nice documentation and you can explore it yourself. Besides that, Pluralsight has a pretty nice course on MongoDB: Introduction to MongoDB by Nuri Halperin. Also, don't miss the Ben's post on the comparison of Map-Reduce in MongoDB and RavenDB.

Tags