After I extracted queries for active menus it was time to decorate them with a cache layer that would speed up indexing batches of recipes. Alas, comparing metrics I quickly realized that something was going wrong.

Still indexing recipes

One of the criteria for a recipe to be indexed is that it needs to be used by at least one active (published) menu. When all requirements are met, information from the active menus are collected and stored together with the recipe, such as product type and menu week, among other things. Menus have up to 10 recipes in some countries, so the same menus are often loaded over and over during the indexing. In order to avoid this, my idea was to create a decorator that would load all menus for a given country, then filter on demand, with a cunning use of array_filter() and foreach. Of course, I would record indexing times along the way and look for stunning gains.

For this experiment I measured two countries: one recent with ~600 recipes, CH; and one older with ~2000 recipes, DE. Here are the initial metrics, with no caching:

CH: NO CACHING
Created: 436, Updated: 0, Deleted: 0, Skipped: 154 — Memory peak: 64 MB — in 26.109 secs
Created: 436, Updated: 0, Deleted: 0, Skipped: 154 — Memory peak: 64 MB — in 25.303 secs
Created: 436, Updated: 0, Deleted: 0, Skipped: 154 — Memory peak: 64 MB — in 25.097 secs

DE: NO CACHING
Created: 1847, Updated: 0, Deleted: 0, Skipped: 70 — Memory peak: 60 MB — in 98.836 secs
Created: 1847, Updated: 0, Deleted: 0, Skipped: 70 — Memory peak: 60 MB — in 97.405 secs
Created: 1847, Updated: 0, Deleted: 0, Skipped: 70 — Memory peak: 60 MB — in 97.225 secs

And this was my glorious decorator (excerpt):

<?php

// …

class ActiveMenuQueriesWithCaching implements ActiveMenuQueries
{
    // …

    /**
     * @param Recipe $recipe
     *
     * @return Menu[]
     */
    public function findByRecipe(Recipe $recipe): array
    {
        $recipeId = $recipe->getId();

        return array_values(array_filter(
            $this->findByCountry(Country::from($recipe->getCountry())),
            function (Menu $menu) use ($recipeId) {
                foreach ($menu->getCourses() as $course) {
                    if ($course->getRecipeId() === $recipeId) {
                        return true;
                    }
                }
                return false;
            }
        ));
    }

    public function existsForRecipe(Recipe $recipe): bool
    {
        $recipeId = $recipe->getId();

        foreach ($this->findByCountry(Country::from($recipe->getCountry())) as $menu) {
            foreach ($menu->getCourses() as $course) {
                if ($course->getRecipeId() === $recipeId) {
                    return true;
                }
            }
        }
        return false;
    }
}

I ran the indexing for CH and was mildly disappointed, these are not the gains I was looking for:

CH: WITH CACHING
Created: 436, Updated: 0, Deleted: 0, Skipped: 154 — Memory peak: 54 MB — in 25.995 secs
Created: 436, Updated: 0, Deleted: 0, Skipped: 154 — Memory peak: 54 MB — in 24.413 secs
Created: 436, Updated: 0, Deleted: 0, Skipped: 154 — Memory peak: 54 MB — in 24.493 secs

One second gain!? That's disappointing alright, and it got way worse as I ran the indexing for DE. As it turned out, querying MongoDB and hydrating entities was faster than filtering stuff in PHP userland, like two times faster.

DE: WITH CACHING
Created: 1847, Updated: 0, Deleted: 0, Skipped: 70 — Memory peak: 66 MB — in 168.210 secs
Created: 1847, Updated: 0, Deleted: 0, Skipped: 70 — Memory peak: 66 MB — in 168.363 secs
Created: 1847, Updated: 0, Deleted: 0, Skipped: 70 — Memory peak: 66 MB — in 165.642 secs

Remember the initial measure was ~98 secs? :sad-face:

Back to the drawing board

I was pretty bummed that my ingenious in-memory search was performing so poorly, but I was not done typing. It was apparent that I had to get rid of these array_filter() and foreach stuff. Since I'm looking for active menus attached to a recipe, I decided to index menus by recipe id, and came up with the following implementation (excerpt):

<?php

// …

class ActiveMenuQueriesWithCaching implements ActiveMenuQueriesContract
{
    // …

    /**
     * @param Recipe $recipe
     *
     * @return Menu[]
     */
    public function findByRecipe(Recipe $recipe): array
    {
        $country = $recipe->getCountry();
        $byRecipe = &$this->byRecipeCache[$country];

        if (empty($byRecipe)) {
            $byRecipe = $this->buildByRecipe(Country::from($country));
        }

        return $byRecipe[$recipe->getId()] ?? [];
    }

    public function existsForRecipe(Recipe $recipe): bool
    {
        $menus = $this->findByRecipe($recipe);

        return !empty($menus);
    }

    /**
     * @param Country $country
     *
     * @return Menu[]
     */
    private function buildByRecipe(Country $country): array
    {
        $byRecipe = [];

        foreach ($this->findByCountry($country) as $menu) {
            foreach ($menu->getCourses() as $course) {
                $recipeId = $course->getRecipeId();

                if (empty($recipeId)) {
                    continue;
                }

                $byRecipe[$recipeId][] = $menu;
            }
        }

        return $byRecipe;
    }
}

I ran the indexing for CH and DE and got the following results:

CH: WITH CACHING, INDEXING
Created: 436, Updated: 0, Deleted: 0, Skipped: 154 — Memory peak: 56 MB — in 18.748 secs
Created: 436, Updated: 0, Deleted: 0, Skipped: 154 — Memory peak: 56 MB — in 18.913 secs
Created: 436, Updated: 0, Deleted: 0, Skipped: 154 — Memory peak: 56 MB — in 18.969 secs

DE: WITH CACHING, INDEXING
Created: 1847, Updated: 0, Deleted: 0, Skipped: 70 — Memory peak: 68 MB — in 74.877 secs
Created: 1847, Updated: 0, Deleted: 0, Skipped: 70 — Memory peak: 68 MB — in 75.274 secs
Created: 1847, Updated: 0, Deleted: 0, Skipped: 70 — Memory peak: 68 MB — in 74.537 secs

~6 seconds saved for CH and ~23 for DE. Yay!

tl;dr

The morale of the story is that whatever you try to optimize, you need to collect metrics along the way to measure if what your are doing is actually effective.