Errors when indexing custom content providers with EPiServer Find

Update december 2016: Johan Bergqvist at Episerver released a blog post on different Episerver Find exception types, check it out at http://world.episerver.com/blogs/Jonas-Bergqvist/Dates/2016/12/exceptions-in-find/

I was issued to solve some indexing issues with a MediaImageContentProvider that inherited from EPiServer.Core.ContentProvider. The indexer threw errors like this

EPiServer.Logging.Compatibility.LogManager+CompatibilityWrapper.Error - DEV02: An exception occurred while indexing content 9444__mediaimageprovider: Object reference not set to an instance of an object. 
System.NullReferenceException: Object reference not set to an instance of an object.
   at lambda_method(Closure , ILocalizable )
   at EPiServer.Find.DelegateValueProvider`2.GetValue(Object target)
   at Newtonsoft.Json.Serialization.JsonSerializerInternalWriter.CalculatePropertyValues(JsonWriter writer, Object value, JsonContainerContract contract, JsonProperty member, JsonProperty property, JsonContract& memberContract, Object& memberValue)
   at Newtonsoft.Json.Serialization.JsonSerializerInternalWriter.SerializeObject(JsonWriter writer, Object value, JsonObjectContract contract, JsonProperty member, JsonContainerContract collectionContract, JsonProperty containerProperty)
   at Newtonsoft.Json.Serialization.JsonSerializerInternalWriter.Serialize(JsonWriter jsonWriter, Object value, Type objectType)
   at EPiServer.Find.Api.BulkActionConverter.WriteJson(JsonWriter writer, Object value, JsonSerializer serializer)
   at Newtonsoft.Json.Serialization.JsonSerializerInternalWriter.SerializeConvertable(JsonWriter writer, JsonConverter converter, Object value, JsonContract contract, JsonContainerContract collectionContract, JsonProperty containerProperty)
   at Newtonsoft.Json.Serialization.JsonSerializerInternalWriter.Serialize(JsonWriter jsonWriter, Object value, Type objectType)
   at Newtonsoft.Json.JsonSerializer.SerializeInternal(JsonWriter jsonWriter, Object value, Type objectType)
   at EPiServer.Find.Json.Serializer.SerializeToTextWriter(JsonSerializer serializer, Object value, TextWriter textWriter)
   at EPiServer.Find.Json.Serializer.SerializeObjectsToJsonRequest(JsonSerializer serializer, IJsonRequest jsonRequest, IEnumerable values)
   at EPiServer.Find.Api.BulkCommand.Execute()
   at EPiServer.Find.Cms.ContentIndexer.IndexWithRetry(IContent[] contents, Int32 maxRetries)
   at EPiServer.Find.Cms.ContentIndexer.Index(IEnumerable`1 content, IndexOptions options)
   at EPiServer.Find.Cms.ContentIndexer.IndexBatch(IEnumerable`1 content, Action`1 statusAction, Int32& numberOfContentErrors, Int32& indexingCount)

This doesn't say much about what the error actually was. After digging around a while I found the following EPiServer Find method that would reveal the actual field causing the null reference error

private static void SerializeToTextWriter(JsonSerializer serializer, object value, TextWriter textWriter)
{
   using (MaxDepthJsonWriter writer = (textWriter is IBookmarkableWriter) ? new BookmarkableMaxDepthJsonWriter(textWriter, serializer.get_MaxDepth()) : new MaxDepthJsonWriter(textWriter, serializer.get_MaxDepth()))
   {
       string loopMember = null;
       Type typeLoopDetectedIn = null;
       Action<object, ErrorEventArgs> action = delegate (object sender, ErrorEventArgs eventArgs) {
           ErrorContext context = eventArgs.get_ErrorContext();
           if (context.get_Member() != null)
           {
               loopMember = context.get_Member().ToString();
           }
           if (context.get_OriginalObject() != null)
           {
               typeLoopDetectedIn = context.get_OriginalObject().GetType();
           }
       };
       try
       {
           serializer.add_Error(new EventHandler<ErrorEventArgs>(action.Invoke));
           serializer.Serialize(writer, value);
       }
       catch (JsonSerializationException exception)
       {
           if (exception.Message.Contains("Self referencing loop detected"))
           {
               throw new SelfReferencingLoopException(loopMember, typeLoopDetectedIn, value, exception);
           }
           throw;
       }
       finally
       {
           serializer.remove_Error(new EventHandler<ErrorEventArgs>(action.Invoke));
       }
   }
}

To solve this error

Setting the MasterLanguage property in the method returning the data, in my case the LoadContent method in the class inheriting the ContentProvider class. This can be done in a few different ways.

If you have a custom content type ensure it implements and fills the following properties

public virtual CultureInfo Language { get; set; }

public IEnumerable<CultureInfo> ExistingLanguages { get; set; }

public CultureInfo MasterLanguage { get; set; }

More in this at http://fellow.aagaardrasmussen.dk/2016/07/31/how-to-make-sure-your-custom-contentprovider-supports-machine-translation-via-translation-com/

If you are using PageData objects in your content provider

protected override IContent LoadContent(ContentReference contentLink, ILanguageSelector languageSelector)
{
    // ...
    pageData.MasterLanguage = new CultureInfo("en");     // EPiServer.Globalization.ContentLanguage.SpecificCulture.TwoLetterISOLanguageName;
    // ...
}

If you are using IContent 

protected override IContent LoadContent(ContentReference contentLink, ILanguageSelector languageSelector)
{
    // ...
    // content is IContent

    var langs = new List<CultureInfo>();
    langs.Add(new CultureInfo("en"); // ensure to set this properly
    ILocalizable localizable = content as ILocalizable;
    localizable.ExistingLanguages = langs;

    // ensures master language support in EPiServer Find
    localizable.MasterLanguage = new CultureInfo("en"); // ensure to set this properly
    localizable.Language = new CultureInfo("en"); // ensure to set this properly

    // ...
}

When nothing works

In one of the providers I worked with none of the above worked. The object returned was a PageData object filled as IContent. In this case I had to override the default MasterLanguage property in the PageData declaration

/// <summary>
/// MasterLanguage override to enable MasterLanguage support in EPiServer Find
/// </summary>
private CultureInfo _masterLangage = null;
public override CultureInfo MasterLanguage
{
    get
    {
        return _masterLangage ?? new CultureInfo(base.LanguageBranch);
    }
    set
    {
        _masterLangage = value;
    }
}

Lesson learned

Always ensure that the MasterLangage property on your indexable ContentTypes have the correct value and aren't null.