Compiled Experience Windows Platform Development http://compiledexperience.com en Wed, 05 Jun 2019 22:13:32 +0000 Wed, 05 Jun 2019 22:13:32 +0000 A better approach to GraphQL renames <p>As it typically happens I write a post on how I do something, it gets some decent traffic and then I find a better way to do it.</p> <p>In this case I’m referring to a post from a few weeks ago about <a href="/blog/posts/stitched-graphql-rename">Handling name collisions in GraphQL schema stitching</a>, in that post I highlighted a way to use <code class="highlighter-rouge">AddMergedDocumentRewriter</code> to rewrite the merged schema document. It turns out there’s a better way…</p> <p><a href="https://hotchocolate.io/">Hot Chocolate</a> supports an <code class="highlighter-rouge">ITypeRewriter</code> interface that when registered will be called for each type in the merged schema. Our <code class="highlighter-rouge">RenameTypeRewriter</code> looks like the following which is a lot cleaner in my opinion.</p> <div class="language-csharp highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">public</span> <span class="k">class</span> <span class="nc">RenameTypeRewriter</span> <span class="p">:</span> <span class="n">ITypeRewriter</span> <span class="p">{</span> <span class="k">public</span> <span class="n">ITypeDefinitionNode</span> <span class="nf">Rewrite</span><span class="p">(</span><span class="n">ISchemaInfo</span> <span class="n">schema</span><span class="p">,</span> <span class="n">ITypeDefinitionNode</span> <span class="n">typeDefinition</span><span class="p">)</span> <span class="p">{</span> <span class="kt">var</span> <span class="n">renameDirective</span> <span class="p">=</span> <span class="n">typeDefinition</span><span class="p">.</span><span class="n">Directives</span><span class="p">.</span><span class="nf">SingleOrDefault</span><span class="p">(</span><span class="n">d</span> <span class="p">=&gt;</span> <span class="n">d</span><span class="p">.</span><span class="n">Name</span><span class="p">.</span><span class="n">Value</span> <span class="p">==</span> <span class="n">RenameDirectiveType</span><span class="p">.</span><span class="n">DirectiveName</span><span class="p">);</span> <span class="k">if</span> <span class="p">(</span><span class="n">renameDirective</span> <span class="p">!=</span> <span class="k">null</span><span class="p">)</span> <span class="p">{</span> <span class="kt">var</span> <span class="n">newNameArgumment</span> <span class="p">=</span> <span class="n">renameDirective</span><span class="p">.</span><span class="n">Arguments</span><span class="p">.</span><span class="nf">Single</span><span class="p">(</span><span class="n">a</span> <span class="p">=&gt;</span> <span class="n">a</span><span class="p">.</span><span class="n">Name</span><span class="p">.</span><span class="n">Value</span> <span class="p">==</span> <span class="n">RenameDirectiveType</span><span class="p">.</span><span class="n">ArgumentName</span> <span class="p">);</span> <span class="k">if</span> <span class="p">(</span><span class="n">newNameArgumment</span><span class="p">.</span><span class="n">Value</span> <span class="k">is</span> <span class="n">StringValueNode</span> <span class="n">stringValue</span><span class="p">)</span> <span class="p">{</span> <span class="k">return</span> <span class="n">typeDefinition</span><span class="p">.</span><span class="nf">Rename</span><span class="p">(</span><span class="n">stringValue</span><span class="p">.</span><span class="n">Value</span><span class="p">,</span> <span class="n">schema</span><span class="p">.</span><span class="n">Name</span><span class="p">);</span> <span class="p">}</span> <span class="p">}</span> <span class="k">return</span> <span class="n">typeDefinition</span><span class="p">;</span> <span class="p">}</span> <span class="p">}</span> </code></pre></div></div> <p>Instead of our call to <code class="highlighter-rouge">AddMergedDocumentRewriter</code> we now have</p> <div class="language-csharp highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="p">.</span><span class="nf">AddTypeRewriter</span><span class="p">(</span><span class="k">new</span> <span class="nf">RenameTypeRewriter</span><span class="p">())</span> </code></pre></div></div> Tue, 04 Jun 2019 00:00:00 +0000 http://compiledexperience.com/blog/posts/better-rename http://compiledexperience.com/blog/posts/better-rename nigel.sampson@compiledexperience.com (Nigel Sampson) csharp graphql Be careful of lazy dependencies <h2 id="what-are-lazy-dependencies">What are lazy dependencies?</h2> <p>.NET Core has a lot more built in support for dependency injection which is an incredibly useful feature and this post isn’t about not using that, but to think about how and when your depdendencies are created. By dependencies I mean any object being created by your Container which is a pretty broad definition which typically covers controllers, services and more.</p> <p>My definition of a lazy dependency is on that’s created by the container the first time it’s required. I think of it as being orthognal to the lifetime of the dependency as singleton dependecies can be lazy as well as transient ones (the differences is in how often they’re created and not when). For most our services this isn’t an issue as often the creation of our dependency doesn’t execute any significant code, the constructors often look something like the following:</p> <div class="language-csharp highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">public</span> <span class="nf">ProductsController</span><span class="p">(</span><span class="n">IProductRepository</span> <span class="n">repository</span><span class="p">,</span> <span class="n">IEventBus</span> <span class="n">events</span><span class="p">)</span> <span class="p">{</span> <span class="k">this</span><span class="p">.</span><span class="n">repository</span> <span class="p">=</span> <span class="n">repository</span><span class="p">;</span> <span class="k">this</span><span class="p">.</span><span class="n">events</span> <span class="p">=</span> <span class="n">events</span><span class="p">;</span> <span class="p">}</span> </code></pre></div></div> <p>This code is unlikely to fail so I don’t have concerns about <code class="highlighter-rouge">ProductsController</code> being lazy. In fact most of our dependencies are lazy (the one’s that aren’t are usually the ones registered as already instantiated objects into the container) and it’s not a problem.</p> <h2 id="which-ones-should-i-be-careful-of">Which ones should I be careful of?</h2> <p>If a majority of our dependencies are lazy what are the ones I should be careful of? In my opinion it’s the ones that have any significant behaviour in their constructors, this could be reading files or opening connections for example. If our <code class="highlighter-rouge">IEventBus</code> implementation establishes a connection to the messaging solution on instantiation then we need to think about what happens if it fails or takes a significant amount of time.</p> <p>This lazy evaluation means the first request to the <code class="highlighter-rouge">ProductsController</code> will be slower due to paying the cost of connecting to the message bus or may fail if that connection fails.</p> <p>A more concrete example comes from our work implementing a GraphQL server using <a href="https://hotchocolate.io/">Hot Chocolate</a>. In GraphQL frameworks you start with the definition of your GraphQL schema, in Hot Chocolate this is represented by the <code class="highlighter-rouge">ISchema</code> interface and configured at start up. However the actual creation of the schema is lazy and doesn’t happy until the first GraphQL request is received which is a little slower but if I’ve misconfigured my schema I don’t see the error till that first request instead of the application startup.</p> <h2 id="why-failing-fast-can-be-good">Why failing fast can be good</h2> <p>Modern deployment strategies (with and without containers) tend to involve standing up the new verison of the application, moving traffic across to the new version and then tearing down the previous version. This enables a seamless deployment experience for the users of the application. However for this to happen the system needs to be able to determine if the system is healthy and the old version can be torn down. If I create a new version where the schema is misconfigured then what want to see happen is the new version is deployed but at start up fails fast, this can then short circuit the deployment system such that previous version keeps serving traffic.</p> <p>Other examples of things we fail fast on are if we detect that there are pending database migrations, this signals that we’re deploying code that is depending on a database migration that hasn’t been deployed to the database. We never want that code to execute so we fail fast and leave the previous version executing.</p> <h2 id="forcing-evaluation">Forcing evaluation</h2> <p>In the examples above we can fail fast by throwing exceptions (this is what we do when there are pending migrations), but for our GraphQL schema example we have a simple extension method.</p> <div class="language-csharp highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c1">/// &lt;summary&gt;</span> <span class="c1">/// Resolves the GraphQL schema forcing validation and ensures any errors before start up failures</span> <span class="c1">/// &lt;/summary&gt;</span> <span class="k">public</span> <span class="k">static</span> <span class="n">IApplicationBuilder</span> <span class="nf">ResolveGraphQLSchema</span><span class="p">(</span><span class="k">this</span> <span class="n">IApplicationBuilder</span> <span class="n">applicationBuilder</span><span class="p">)</span> <span class="p">{</span> <span class="k">try</span> <span class="p">{</span> <span class="n">applicationBuilder</span><span class="p">.</span><span class="n">ApplicationServices</span><span class="p">.</span><span class="n">GetRequiredService</span><span class="p">&lt;</span><span class="n">ISchema</span><span class="p">&gt;();</span> <span class="k">return</span> <span class="n">applicationBuilder</span><span class="p">;</span> <span class="p">}</span> <span class="k">catch</span> <span class="p">(</span><span class="n">Exception</span> <span class="n">ex</span><span class="p">)</span> <span class="p">{</span> <span class="k">throw</span> <span class="k">new</span> <span class="nf">StartupFailureException</span><span class="p">(</span><span class="s">"Failed to resolve the sitched GraphQL schema"</span><span class="p">,</span> <span class="n">ex</span><span class="p">);</span> <span class="p">}</span> <span class="p">}</span> </code></pre></div></div> <p>and then called during start up.</p> <div class="language-csharp highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">public</span> <span class="k">void</span> <span class="nf">Configure</span><span class="p">(</span><span class="n">IApplicationBuilder</span> <span class="n">app</span><span class="p">,</span> <span class="n">IHostingEnvironment</span> <span class="n">env</span><span class="p">,</span> <span class="n">ILoggerFactory</span> <span class="n">loggerFactory</span><span class="p">,</span> <span class="n">IApplicationLifetime</span> <span class="n">appLifetime</span><span class="p">)</span> <span class="p">{</span> <span class="p">...</span> <span class="n">app</span><span class="p">.</span><span class="nf">ResolveGraphQLSchema</span><span class="p">();</span> <span class="p">...</span> <span class="p">}</span> </code></pre></div></div> <h2 id="summary">Summary</h2> <ul> <li>Most container depdendencies are “lazy”, only instantiated when required.</li> <li>This isn’t a problem when there is no logic in their constructors.</li> <li>When there is we may see errors or slow downs on first requests.</li> <li>Given modern deployment strategies failing fast is a better pattern.</li> <li>We can fore evaluation of these dependencies to move to “fail fast” semantics.</li> </ul> Thu, 23 May 2019 00:00:00 +0000 http://compiledexperience.com/blog/posts/lazy-dependencies http://compiledexperience.com/blog/posts/lazy-dependencies nigel.sampson@compiledexperience.com (Nigel Sampson) csharp graphql Handling name collisions in GraphQL schema stitching <p><strong>GraphQL Schema Stitching</strong> is the mechanism of composing multiple GraphQL schemas together into a single unified schema. This plays strongly into the concept of “back-ends for front-ends” which is building a specialised API designed specifically for the app in question. This “API Gateway” takes the schemas of the micro-services needed for it’s functionality, combines, extends and exposes them to the front end.</p> <p>In the .NET ecosystem the framework <a href="https://hotchocolate.io/">Hot Chocolate</a> has excellent support for stitching together multiple remote schemas as well as customising and extending them. If you’re working with GraphQL in this ecosystem I’d recommend checking it out.</p> <h2 id="name-collisions">Name collisions</h2> <p>GraphQL has no concept of namespacing and enforces unique names on types, so how should we deal with two different schemas exposing types with the same name?</p> <h3 id="pseudo-namespaces">Pseudo-namespaces</h3> <p>Some companies have solved this by introducing a namespace concept by prefixing all types in the schema with the service name. The schema for the Orders service may look like the following. While this works and should guarantee that you have no collisions it cam make for some ugly looking schemas.</p> <div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>type Orders_LineItem { productId; ID! quantity: Int! } type Orders_Order { id: ID! items: [Orders_LineItem!]! } type Query { orders(customerId: ID!): [Orders_Order!]! } </code></pre></div></div> <h3 id="renaming">Renaming</h3> <p>Another approach is to “rename” the type at the stitched schema if your framework supports it, thankfully <a href="https://hotchocolate.io/">Hot Chocolate</a> does. The <code class="highlighter-rouge">IStitchingBuilder</code> lets us rename types as we build our new schema. The following snippet stitches our Customers and Orders schemas and renames the <code class="highlighter-rouge">Customer</code> type in the Orders schema to <code class="highlighter-rouge">CustomerDetails</code>.</p> <div class="language-csharp highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">services</span><span class="p">.</span><span class="nf">AddStitchedSchema</span><span class="p">(</span><span class="n">builder</span> <span class="p">=&gt;</span> <span class="n">builder</span> <span class="p">.</span><span class="nf">AddSchemaFromFile</span><span class="p">(</span><span class="s">"customers"</span><span class="p">,</span> <span class="s">"./schemas/customers.graphql"</span><span class="p">)</span> <span class="p">.</span><span class="nf">AddSchemaFromFile</span><span class="p">(</span><span class="s">"orders"</span><span class="p">,</span> <span class="s">"./schemas/orders.graphql"</span><span class="p">)</span> <span class="p">.</span><span class="nf">RenameType</span><span class="p">(</span><span class="s">"orders"</span><span class="p">,</span> <span class="s">"Customer"</span><span class="p">,</span> <span class="s">"CustomerDetails"</span><span class="p">)</span> </code></pre></div></div> <p>While this works it feels very <em>stringly typed</em> and if you have a number of renames to do would become difficult to maintain. How can we do this better?</p> <p>One extension point provided is <code class="highlighter-rouge">AddMergedDocumentRewriter</code>, this takes a <code class="highlighter-rouge">Func&lt;DocumentNode, DocumentNode&gt;</code> and lets you make any customisations and additions to the schema programmatically. So let’s see if we can build a way to specify the renames in the schema files rather than in code.</p> <p>The question of “Why we’re using <code class="highlighter-rouge">AddSchemaFromFile</code> instead of <code class="highlighter-rouge">AddSchemaFromHttp</code> is a blog post for another day, but it certainlu helps us here.</p> <p>Our end goal will be to remove the <code class="highlighter-rouge">RenameType</code> call and instead inside the <code class="highlighter-rouge">orders.graphql</code> have something like</p> <div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>type Customer @rename(name: "CustomerDetails") { id: ID! firstName: String! ... } </code></pre></div></div> <h2 id="rewriting-the-stitched-schema">Rewriting the stitched schema</h2> <p>First we should define our <code class="highlighter-rouge">rename</code> directive, Directives are the mechanism to add metadata to GraphQL schemas much like C# attributes.</p> <div class="language-csharp highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">public</span> <span class="k">class</span> <span class="nc">RenameDirectiveType</span> <span class="p">:</span> <span class="n">DirectiveType</span> <span class="p">{</span> <span class="k">public</span> <span class="k">const</span> <span class="kt">string</span> <span class="n">DirectiveName</span> <span class="p">=</span> <span class="s">"rename"</span><span class="p">;</span> <span class="k">public</span> <span class="k">const</span> <span class="kt">string</span> <span class="n">ArgumentName</span> <span class="p">=</span> <span class="s">"name"</span><span class="p">;</span> <span class="k">protected</span> <span class="k">override</span> <span class="k">void</span> <span class="nf">Configure</span><span class="p">(</span><span class="n">IDirectiveTypeDescriptor</span> <span class="n">descriptor</span><span class="p">)</span> <span class="p">{</span> <span class="n">descriptor</span><span class="p">.</span><span class="nf">Name</span><span class="p">(</span><span class="n">DirectiveName</span><span class="p">);</span> <span class="n">descriptor</span><span class="p">.</span><span class="nf">Location</span><span class="p">(</span><span class="n">DirectiveLocation</span><span class="p">.</span><span class="n">Object</span><span class="p">);</span> <span class="n">descriptor</span><span class="p">.</span><span class="nf">Argument</span><span class="p">(</span><span class="n">ArgumentName</span><span class="p">)</span> <span class="p">.</span><span class="n">Type</span><span class="p">&lt;</span><span class="n">NonNullType</span><span class="p">&lt;</span><span class="n">NameType</span><span class="p">&gt;&gt;();</span> <span class="p">}</span> <span class="p">}</span> </code></pre></div></div> <p>We can now build our document rewriter to use this directive</p> <blockquote> <p><strong>Update</strong> I’ve found a better way to do the following, documented at <a href="/blog/posts/better-rename"> A better approach to GraphQL renames</a>.</p> </blockquote> <div class="language-csharp highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">public</span> <span class="k">static</span> <span class="k">class</span> <span class="nc">Rewriters</span> <span class="p">{</span> <span class="k">public</span> <span class="k">static</span> <span class="n">DocumentNode</span> <span class="nf">RenameTypes</span><span class="p">(</span><span class="n">DocumentNode</span> <span class="n">document</span><span class="p">)</span> <span class="p">{</span> <span class="kt">var</span> <span class="n">definitions</span> <span class="p">=</span> <span class="k">new</span> <span class="n">List</span><span class="p">&lt;</span><span class="n">IDefinitionNode</span><span class="p">&gt;();</span> <span class="k">foreach</span> <span class="p">(</span><span class="kt">var</span> <span class="n">definition</span> <span class="k">in</span> <span class="n">document</span><span class="p">.</span><span class="n">Definitions</span><span class="p">)</span> <span class="p">{</span> <span class="k">if</span> <span class="p">(</span><span class="n">definition</span> <span class="k">is</span> <span class="n">ObjectTypeDefinitionNode</span> <span class="n">typeDefinition</span><span class="p">)</span> <span class="p">{</span> <span class="kt">var</span> <span class="n">renameDirective</span> <span class="p">=</span> <span class="n">typeDefinition</span><span class="p">.</span><span class="n">Directives</span><span class="p">.</span><span class="nf">SingleOrDefault</span><span class="p">(</span><span class="n">d</span> <span class="p">=&gt;</span> <span class="n">d</span><span class="p">.</span><span class="n">Name</span><span class="p">.</span><span class="n">Value</span> <span class="p">==</span> <span class="n">RenameDirectiveType</span><span class="p">.</span><span class="n">DirectiveName</span><span class="p">);</span> <span class="k">if</span> <span class="p">(</span><span class="n">renameDirective</span> <span class="p">!=</span> <span class="k">null</span><span class="p">)</span> <span class="p">{</span> <span class="kt">var</span> <span class="n">newNameArgumment</span> <span class="p">=</span> <span class="n">renameDirective</span><span class="p">.</span><span class="n">Arguments</span><span class="p">.</span><span class="nf">Single</span><span class="p">(</span><span class="n">a</span> <span class="p">=&gt;</span> <span class="n">a</span><span class="p">.</span><span class="n">Name</span><span class="p">.</span><span class="n">Value</span> <span class="p">==</span> <span class="n">RenameDirectiveType</span><span class="p">.</span><span class="n">ArgumentName</span><span class="p">);</span> <span class="k">if</span> <span class="p">(</span><span class="n">newNameArgumment</span><span class="p">.</span><span class="n">Value</span> <span class="k">is</span> <span class="n">StringValueNode</span> <span class="n">stringValue</span><span class="p">)</span> <span class="p">{</span> <span class="n">definitions</span><span class="p">.</span><span class="nf">Add</span><span class="p">(</span><span class="n">typeDefinition</span><span class="p">.</span><span class="nf">WithName</span><span class="p">(</span><span class="k">new</span> <span class="nf">NameNode</span><span class="p">(</span><span class="n">stringValue</span><span class="p">.</span><span class="n">Value</span><span class="p">)));</span> <span class="k">continue</span><span class="p">;</span> <span class="p">}</span> <span class="p">}</span> <span class="p">}</span> <span class="n">definitions</span><span class="p">.</span><span class="nf">Add</span><span class="p">(</span><span class="n">definition</span><span class="p">);</span> <span class="p">}</span> <span class="k">return</span> <span class="n">document</span><span class="p">.</span><span class="nf">WithDefinitions</span><span class="p">(</span><span class="n">definitions</span><span class="p">);</span> <span class="p">}</span> <span class="p">}</span> </code></pre></div></div> <p>This looks pretty complex but isn’t too bad. It’s important to note that <code class="highlighter-rouge">DocumentNode</code> is immutable, so we build a new one with the renamed types, the <code class="highlighter-rouge">WithDefinitions</code> helper does this, essentially returning a new <code class="highlighter-rouge">DocumentNode</code> which is the same as the old but with different type definitions.</p> <p>The rest of the code simply loops over all the type definitions in the document and examines the object type ones closer (if you wanted to support renaming fields this becomes a little more complex), if this object type definition has the rename directive then get the argument value and add the renamed definition to the list.</p> <p>We can now use our rewriter as follows</p> <div class="language-csharp highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">services</span><span class="p">.</span><span class="nf">AddStitchedSchema</span><span class="p">(</span><span class="n">builder</span> <span class="p">=&gt;</span> <span class="n">builder</span> <span class="p">.</span><span class="nf">AddSchemaFromFile</span><span class="p">(</span><span class="s">"customers"</span><span class="p">,</span> <span class="s">"./schemas/customers.graphql"</span><span class="p">)</span> <span class="p">.</span><span class="nf">AddSchemaFromFile</span><span class="p">(</span><span class="s">"orders"</span><span class="p">,</span> <span class="s">"./schemas/orders.graphql"</span><span class="p">)</span> <span class="p">.</span><span class="nf">AddMergedDocumentRewriter</span><span class="p">(</span><span class="n">Rewriters</span><span class="p">.</span><span class="n">RenameTypes</span><span class="p">)</span> </code></pre></div></div> <p>We can now handle all our renames in the schema files rather than using magic strings in code.</p> <h2 id="summary">Summary</h2> <ul> <li>GraphQL schema stiching combines multiple schemas in single exposed schema.</li> <li>Given the lack of namespaces in the specification we need to handle type name collisions between the schema.</li> <li>While Hot Chocolate supports renaming types the mechanism leads to a lot of hard to maintain code.</li> <li>Building our own document rewriter lets us do directive based renames.</li> </ul> <p>Hope this helps someone.</p> Wed, 08 May 2019 00:00:00 +0000 http://compiledexperience.com/blog/posts/stitched-graphql-rename http://compiledexperience.com/blog/posts/stitched-graphql-rename nigel.sampson@compiledexperience.com (Nigel Sampson) csharp graphql My git aliases <p>I’ve seen a few posts around lately with people discussing how they work with git during day to day development and I thought I’d share mine.</p> <p>I use a combination of <a href="https://desktop.github.com/">GitHub Desktop</a> and the command line. I prefer a UI for crafting my commits, being able to select which files and / or lines I’d like to include is incredibly easy in the UI.</p> <p>For everything else such as rebasing, pulling etc I use the command line, usually Powershell inside <a href="https://cmder.net/">Cmder</a> and using <a href="https://github.com/dahlbyk/posh-git">posh-git</a>.</p> <p>Below are the aliases I use reasonably consistently, one theme through these is that I’m not optimising for keystrokes, but “brain space”.</p> <div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>graph = log --oneline --graph --decorate --all branches = branch -a tags = tag stashes = stash list unstage = reset -q HEAD -- discard = checkout -- uncommit = reset --mixed HEAD~ amend = commit --amend --no-edit nevermind = !git reset --hard HEAD &amp;&amp; git clean -xdf remotes = remote -v fixup = commit --fixup overwrite = push --force-with-lease rewrite="!f() { \ COMMIT_COUNT=$(git rev-list --count HEAD ^master); \ git rebase -i --autosquash HEAD~$COMMIT_COUNT; \ };f" publish="!f() { \ BRANCH_NAME=$(git rev-parse --abbrev-ref HEAD); \ git push -u origin $BRANCH_NAME; \ };f" browse = "!f() { \ REPO_URL=$(git config remote.origin.url); \ start ${REPO_URL%%.git}; \ };f" pr = "!f() { \ REPO_URL=$(git config remote.origin.url); \ BRANCH_NAME=$(git rev-parse --abbrev-ref HEAD); \ start \"${REPO_URL%%.git}/compare/${BRANCH_NAME}?expand=1\"; \ };f" jira = "!f() { \ BRANCH_NAME=$(git rev-parse --abbrev-ref HEAD); \ start \"https://pushpay.atlassian.net/browse/${BRANCH_NAME}\"; \ };f" </code></pre></div></div> <p>Hope others find these useful.</p> Fri, 05 Apr 2019 00:00:00 +0000 http://compiledexperience.com/blog/posts/git-alias-2019 http://compiledexperience.com/blog/posts/git-alias-2019 nigel.sampson@compiledexperience.com (Nigel Sampson) git Guarding against N+1 issues in GraphQL <p>The <strong>N+1 query problem</strong> is a common one to encounter during software development, particularly with ORMs (Object Relational Mappers) and their capabilities around lazy loading. A quick example looks like</p> <div class="language-csharp highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kt">var</span> <span class="n">orders</span> <span class="p">=</span> <span class="nf">GetOrders</span><span class="p">();</span> <span class="k">foreach</span><span class="p">(</span><span class="kt">var</span> <span class="n">order</span> <span class="k">in</span> <span class="n">orders</span><span class="p">)</span> <span class="p">{</span> <span class="n">Console</span><span class="p">.</span><span class="nf">WriteLine</span><span class="p">(</span><span class="s">$"</span><span class="p">{</span><span class="n">order</span><span class="p">.</span><span class="n">Customer</span><span class="p">.</span><span class="n">Name</span><span class="p">}</span><span class="s"> made an order on </span><span class="p">{</span><span class="n">order</span><span class="p">.</span><span class="n">CreatedOn</span><span class="p">}</span><span class="s">"</span><span class="p">);</span> <span class="p">}</span> </code></pre></div></div> <p>Unless you’re careful, the lazy loading of the <code class="highlighter-rouge">Customer</code> of the <code class="highlighter-rouge">Order</code> results in a database query, for 100 orders this would result in 101 database calls, one per <code class="highlighter-rouge">Customer</code> and one extra for the orders (hence the name N+1).</p> <p>I won’t go into the ways we can solve this in ORMs as this is a well documented problem with a myriad of approaches to avoid it.</p> <h2 id="graphql-and-n1">GraphQL and N+1</h2> <p>When we look at a typical GraphQL query we can see the makings of the same problem.</p> <div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>query { orders { createdOn customer { name } } } </code></pre></div></div> <p>Whether the above query results in N+1 database queries is completely up to the the implementation of the resolvers. Most GraphQL server frameworks support the idea of a <code class="highlighter-rouge">DataLoader</code>. A Dataloader allows you to batch the <code class="highlighter-rouge">Customer</code> requests into a single query and put the correct <code class="highlighter-rouge">Customer</code> into the correct location of the results. For better examples you can view the documentation for <a href="https://hotchocolate.io/docs/dataloaders">Hot Chocolate</a>, a popular .NET GraphQL server framework. This only works when the underlying data source supports a performant batch operation (in these case getting a list of customers based on a list of ids).</p> <h2 id="when-dataloader-wont-work">When DataLoader won’t work</h2> <p>Sometimes however this batch operation isn’t supported and we’re left with our N+1 performance problem, what can we do?</p> <p>Let’s assume our Orders and Customers data live in different microservices and that Customers microservice doesn’t support a batch operation for requesting a list of Customers.</p> <p>In that case a query like the above one wouldn’t be performant, resulting in N requests to the underlying Customer microservice, however a query like the following would only result in a single call.</p> <div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>query { order("b3JkZXI6NDI=") { createdOn customer { name } } } </code></pre></div></div> <p>So how do we ensure one is valid, but one is invalid? The easiest way would be to define different <code class="highlighter-rouge">Order</code> types in our schema, one that has <code class="highlighter-rouge">customer</code> field and one doesn’t and never expose the former in a list field. While it works it would cause a long term maintenance problem as we struggle to keep these types in sync and doesn’t create a particularly nice graph.</p> <p>Instead we’re going to play with the <a href="https://hotchocolate.io/docs/validation-rule">validation rules</a> feature of Hot Chocolate. We’ll create a rule that the <code class="highlighter-rouge">customer</code> field can only be included in a query where none of its parent fields are a list. This would ensure the latter query is valid while the former is invalid.</p> <p>One really nice feature about validation rules are that they’re done during the parsing of the query document. This means the validation result can be cached along with the parsing result. This way if you’re sending constant queries (where only the variables are different) then the validation rule won’t be executed that often.</p> <h2 id="creating-the-directive">Creating the Directive</h2> <p>First we need to create a directive, these function in much the same way as C# attributes, a way to attach metadata to our GraphQL schema that other code can introspect and make use of. In this case we’ll define an <code class="highlighter-rouge">includeOnce</code>.</p> <div class="language-csharp highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">public</span> <span class="k">class</span> <span class="nc">IncludeOnceDirectiveType</span> <span class="p">:</span> <span class="n">DirectiveType</span> <span class="p">{</span> <span class="k">protected</span> <span class="k">override</span> <span class="k">void</span> <span class="nf">Configure</span><span class="p">(</span><span class="n">IDirectiveTypeDescriptor</span> <span class="n">descriptor</span><span class="p">)</span> <span class="p">{</span> <span class="n">descriptor</span><span class="p">.</span><span class="nf">Name</span><span class="p">(</span><span class="s">"includeOnce"</span><span class="p">);</span> <span class="n">descriptor</span><span class="p">.</span><span class="nf">Location</span><span class="p">(</span><span class="n">DirectiveLocation</span><span class="p">.</span><span class="n">FieldDefinition</span><span class="p">);</span> <span class="p">}</span> <span class="p">}</span> </code></pre></div></div> <p>This directive doesn’t have any behavior and simply functions as a “marker” for our validation rule. If we’re doing schema first development we register the directive in the schema.</p> <div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>type Order { ... customer: Customer @includeOnce } </code></pre></div></div> <p>While in code first it would be in the field definition</p> <div class="language-csharp highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">descriptor</span><span class="p">.</span><span class="nf">Field</span><span class="p">(</span><span class="s">"customer"</span><span class="p">)</span> <span class="p">.</span><span class="n">Directive</span><span class="p">&lt;</span><span class="n">IncludeOnceDirectiveType</span><span class="p">&gt;()</span> <span class="p">.</span><span class="n">Type</span><span class="p">&lt;</span><span class="n">NonNullType</span><span class="p">&lt;</span><span class="n">CustomerType</span><span class="p">&gt;&gt;()</span> <span class="p">...</span> </code></pre></div></div> <p>The directive also needs to be registered with the schema itself.</p> <div class="language-csharp highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">c</span><span class="p">.</span><span class="n">RegisterDirective</span><span class="p">&lt;</span><span class="n">IncludeOnceDirectiveType</span><span class="p">&gt;();</span> </code></pre></div></div> <h2 id="creating-the-validation-rule">Creating the validation rule</h2> <p>The implementation of the validation rule can be quite complex, the code below illustrates the concept but doesn’t cover some of the more complex cases (such as mutations). For a more fully featured example I suggest the <a href="https://github.com/ChilliCream/hotchocolate/blob/master/src/Core/Core/Validation/MaxComplexityRule.cs"><code class="highlighter-rouge">MaxComplexityRule</code></a> and <a href="https://github.com/ChilliCream/hotchocolate/blob/master/src/Core/Core/Validation/MaxComplexityVisitor.cs"><code class="highlighter-rouge">MaxComplexityVisitor</code></a> from the core repository.</p> <p>What we need to do is create a <code class="highlighter-rouge">QuerySyntaxWalker</code> that supports the traversal of the AST (abstract syntax tree), if you’ve done anything with Roslyn some of this will look familiar.</p> <p>It can look complicated, but we’re visiting each field within the query and maintaining some context from the parent fields we’ve visited. If we find a field whose definition has the <code class="highlighter-rouge">includeOnce</code> directive then we examine the list of parent fields, if any are a list type then we report an error.</p> <div class="language-csharp highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">public</span> <span class="k">class</span> <span class="nc">IncludeOnceVisitor</span> <span class="p">:</span> <span class="n">QuerySyntaxWalker</span><span class="p">&lt;</span><span class="n">IncludeOnceContext</span><span class="p">&gt;</span> <span class="p">{</span> <span class="k">protected</span> <span class="k">override</span> <span class="kt">bool</span> <span class="n">VisitFragmentDefinitions</span> <span class="p">=&gt;</span> <span class="k">false</span><span class="p">;</span> <span class="k">protected</span> <span class="k">override</span> <span class="k">void</span> <span class="nf">VisitField</span><span class="p">(</span><span class="n">FieldNode</span> <span class="n">node</span><span class="p">,</span> <span class="n">IncludeOnceContext</span> <span class="n">context</span><span class="p">)</span> <span class="p">{</span> <span class="kt">var</span> <span class="n">newContext</span> <span class="p">=</span> <span class="n">context</span><span class="p">;</span> <span class="k">if</span> <span class="p">(</span><span class="n">context</span><span class="p">.</span><span class="n">TypeContext</span> <span class="k">is</span> <span class="n">IComplexOutputType</span> <span class="n">type</span> <span class="p">&amp;&amp;</span> <span class="n">type</span><span class="p">.</span><span class="n">Fields</span><span class="p">.</span><span class="nf">TryGetField</span><span class="p">(</span><span class="n">node</span><span class="p">.</span><span class="n">Name</span><span class="p">.</span><span class="n">Value</span><span class="p">,</span> <span class="k">out</span> <span class="n">IOutputField</span> <span class="n">fieldDefinition</span><span class="p">))</span> <span class="p">{</span> <span class="n">newContext</span> <span class="p">=</span> <span class="n">newContext</span><span class="p">.</span><span class="nf">WithField</span><span class="p">(</span><span class="n">fieldDefinition</span><span class="p">);</span> <span class="k">if</span> <span class="p">(</span><span class="n">fieldDefinition</span><span class="p">.</span><span class="n">Type</span><span class="p">.</span><span class="nf">NamedType</span><span class="p">()</span> <span class="k">is</span> <span class="n">IComplexOutputType</span> <span class="n">ct</span><span class="p">)</span> <span class="p">{</span> <span class="n">newContext</span> <span class="p">=</span> <span class="n">newContext</span><span class="p">.</span><span class="nf">SetTypeContext</span><span class="p">(</span><span class="n">ct</span><span class="p">);</span> <span class="k">if</span> <span class="p">(</span><span class="n">fieldDefinition</span><span class="p">.</span><span class="n">Directives</span><span class="p">.</span><span class="nf">Contains</span><span class="p">(</span><span class="s">"includeOnce"</span><span class="p">))</span> <span class="p">{</span> <span class="kt">var</span> <span class="n">includedMoreThanOnce</span> <span class="p">=</span> <span class="n">context</span><span class="p">.</span><span class="n">FieldPath</span><span class="p">.</span><span class="nf">Any</span><span class="p">(</span><span class="n">f</span> <span class="p">=&gt;</span> <span class="n">f</span><span class="p">.</span><span class="n">Type</span><span class="p">.</span><span class="nf">IsListType</span><span class="p">());</span> <span class="k">if</span> <span class="p">(</span><span class="n">includedMoreThanOnce</span><span class="p">)</span> <span class="p">{</span> <span class="n">newContext</span><span class="p">.</span><span class="nf">ReportError</span><span class="p">(</span><span class="k">new</span> <span class="nf">ValidationError</span><span class="p">(</span><span class="s">$"The field </span><span class="p">{</span><span class="n">node</span><span class="p">.</span><span class="n">Name</span><span class="p">}</span><span class="s"> can only be includes once in a query"</span><span class="p">,</span> <span class="n">node</span><span class="p">));</span> <span class="p">}</span> <span class="p">}</span> <span class="p">}</span> <span class="k">base</span><span class="p">.</span><span class="nf">VisitField</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="n">newContext</span><span class="p">);</span> <span class="p">}</span> <span class="p">}</span> <span class="p">}</span> <span class="k">public</span> <span class="k">class</span> <span class="nc">IncludeOnceContext</span> <span class="p">{</span> <span class="k">public</span> <span class="nf">IncludeOnceContext</span><span class="p">(</span><span class="n">ISchema</span> <span class="n">schema</span><span class="p">,</span> <span class="n">Action</span><span class="p">&lt;</span><span class="n">IError</span><span class="p">&gt;</span> <span class="n">reportError</span><span class="p">)</span> <span class="p">{</span> <span class="n">Schema</span> <span class="p">=</span> <span class="n">schema</span><span class="p">;</span> <span class="n">FieldPath</span> <span class="p">=</span> <span class="k">new</span> <span class="n">List</span><span class="p">&lt;</span><span class="n">IOutputField</span><span class="p">&gt;();</span> <span class="n">ReportError</span> <span class="p">=</span> <span class="n">reportError</span><span class="p">;</span> <span class="p">}</span> <span class="k">protected</span> <span class="nf">IncludeOnceContext</span><span class="p">(</span><span class="n">ISchema</span> <span class="n">schema</span><span class="p">,</span> <span class="n">IEnumerable</span><span class="p">&lt;</span><span class="n">IOutputField</span><span class="p">&gt;</span> <span class="n">fieldPath</span><span class="p">,</span> <span class="n">Action</span><span class="p">&lt;</span><span class="n">IError</span><span class="p">&gt;</span> <span class="n">reportError</span><span class="p">)</span> <span class="p">{</span> <span class="n">Schema</span> <span class="p">=</span> <span class="n">schema</span><span class="p">;</span> <span class="n">FieldPath</span> <span class="p">=</span> <span class="n">fieldPath</span><span class="p">.</span><span class="nf">ToList</span><span class="p">();</span> <span class="n">ReportError</span> <span class="p">=</span> <span class="n">reportError</span><span class="p">;</span> <span class="p">}</span> <span class="k">public</span> <span class="n">Action</span><span class="p">&lt;</span><span class="n">IError</span><span class="p">&gt;</span> <span class="n">ReportError</span> <span class="p">{</span> <span class="k">get</span><span class="p">;</span> <span class="p">}</span> <span class="k">public</span> <span class="n">IList</span><span class="p">&lt;</span><span class="n">IOutputField</span><span class="p">&gt;</span> <span class="n">FieldPath</span> <span class="p">{</span> <span class="k">get</span><span class="p">;</span> <span class="p">}</span> <span class="k">public</span> <span class="n">ISchema</span> <span class="n">Schema</span> <span class="p">{</span> <span class="k">get</span><span class="p">;</span> <span class="p">}</span> <span class="k">public</span> <span class="n">INamedOutputType</span> <span class="n">TypeContext</span> <span class="p">{</span> <span class="k">get</span><span class="p">;</span> <span class="k">protected</span> <span class="k">set</span><span class="p">;</span> <span class="p">}</span> <span class="k">public</span> <span class="n">IncludeOnceContext</span> <span class="nf">WithField</span><span class="p">(</span><span class="n">IOutputField</span> <span class="n">field</span><span class="p">)</span> <span class="p">{</span> <span class="k">return</span> <span class="k">new</span> <span class="nf">IncludeOnceContext</span><span class="p">(</span><span class="n">Schema</span><span class="p">,</span> <span class="n">FieldPath</span><span class="p">.</span><span class="nf">Concat</span><span class="p">(</span><span class="k">new</span><span class="p">[]</span> <span class="p">{</span> <span class="n">field</span> <span class="p">}),</span> <span class="n">ReportError</span><span class="p">);</span> <span class="p">}</span> <span class="k">public</span> <span class="n">IncludeOnceContext</span> <span class="nf">SetTypeContext</span><span class="p">(</span><span class="n">INamedOutputType</span> <span class="n">typeContext</span><span class="p">)</span> <span class="p">{</span> <span class="k">return</span> <span class="k">new</span> <span class="nf">IncludeOnceContext</span><span class="p">(</span><span class="n">Schema</span><span class="p">,</span> <span class="n">FieldPath</span><span class="p">,</span> <span class="n">ReportError</span><span class="p">)</span> <span class="p">{</span> <span class="n">TypeContext</span> <span class="p">=</span> <span class="n">typeContext</span> <span class="p">};</span> <span class="p">}</span> <span class="p">}</span> </code></pre></div></div> <p>After all this, the validation rule itself is pretty straightforward. Create a visitor, explore the document, then report if there are any errors.</p> <div class="language-csharp highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">public</span> <span class="k">class</span> <span class="nc">IncludeOnceValidationRule</span> <span class="p">:</span> <span class="n">IQueryValidationRule</span> <span class="p">{</span> <span class="k">private</span> <span class="k">static</span> <span class="k">readonly</span> <span class="n">IncludeOnceVisitor</span> <span class="n">includeOnceVisitor</span> <span class="p">=</span> <span class="k">new</span> <span class="nf">IncludeOnceVisitor</span><span class="p">();</span> <span class="k">public</span> <span class="n">QueryValidationResult</span> <span class="nf">Validate</span><span class="p">(</span><span class="n">ISchema</span> <span class="n">schema</span><span class="p">,</span> <span class="n">DocumentNode</span> <span class="n">queryDocument</span><span class="p">)</span> <span class="p">{</span> <span class="kt">var</span> <span class="n">errors</span> <span class="p">=</span> <span class="k">new</span> <span class="n">List</span><span class="p">&lt;</span><span class="n">IError</span><span class="p">&gt;();</span> <span class="kt">var</span> <span class="n">context</span> <span class="p">=</span> <span class="k">new</span> <span class="nf">IncludeOnceContext</span><span class="p">(</span><span class="n">schema</span><span class="p">,</span> <span class="n">errors</span><span class="p">.</span><span class="n">Add</span><span class="p">).</span><span class="nf">SetTypeContext</span><span class="p">(</span><span class="n">schema</span><span class="p">.</span><span class="n">QueryType</span><span class="p">);</span> <span class="n">includeOnceVisitor</span><span class="p">.</span><span class="nf">Visit</span><span class="p">(</span><span class="n">queryDocument</span><span class="p">,</span> <span class="n">context</span><span class="p">);</span> <span class="k">return</span> <span class="n">errors</span><span class="p">.</span><span class="nf">Any</span><span class="p">()</span> <span class="p">?</span> <span class="k">new</span> <span class="nf">QueryValidationResult</span><span class="p">(</span><span class="n">errors</span><span class="p">)</span> <span class="p">:</span> <span class="n">QueryValidationResult</span><span class="p">.</span><span class="n">OK</span><span class="p">;</span> <span class="p">}</span> <span class="p">}</span> </code></pre></div></div> <p>The one final step is register our new validation rule with the schema</p> <div class="language-csharp highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="p">.</span><span class="nf">AddExecutionConfiguration</span><span class="p">(</span><span class="n">execution</span> <span class="p">=&gt;</span> <span class="n">execution</span> <span class="p">.</span><span class="n">AddValidationRule</span><span class="p">&lt;</span><span class="n">IncludeOnceValidationRule</span><span class="p">&gt;()</span> <span class="p">)</span> </code></pre></div></div> <p>We can now validate that our poorly performing field is only included in queries that will not exacerbate the issue. Hope this helps.</p> Wed, 13 Mar 2019 00:00:00 +0000 http://compiledexperience.com/blog/posts/graphql-n+1 http://compiledexperience.com/blog/posts/graphql-n+1 nigel.sampson@compiledexperience.com (Nigel Sampson) csharp graphql Exporting a GraphQL Schema <p>I’ve been playing around with <a href="https://graphql.org/">GraphQL</a> at work quite a bit lately and starting to put together some thoughts on how it can fit into your application (and more importantly where it doesn’t).</p> <p>One of the things we found we needed everyonce in a while was an export of the schema in the <code class="highlighter-rouge">.graphql</code> format. Most of the time we were definining our schema in a “code first* way using <a href="https://graphql-dotnet.github.io/">GraphQL.NET</a>, so we typically didn’t have this file to work from.</p> <p>However a number of tools / frameworks work well if they can import one of these schemas, a good example is schema stiching using <a href="https://hotchocolate.io/">Hot Chocolate</a>.</p> <p>So how do we export a schema from an already existing <a href="https://graphql-dotnet.github.io/">GraphQL.NET</a> service?</p> <p>Thankfully support comes in terms of the <code class="highlighter-rouge">SchemaPrinter</code> class. This takes an instance of your <code class="highlighter-rouge">Schema</code> that you’ve created by any of the mechanisms the framework supports.</p> <p>Usage looks something like</p> <div class="language-csharp highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">using</span> <span class="p">(</span><span class="kt">var</span> <span class="n">printer</span> <span class="p">=</span> <span class="k">new</span> <span class="nf">SchemaPrinter</span><span class="p">(</span><span class="n">_schema</span><span class="p">,</span> <span class="k">new</span> <span class="n">SchemaPrinterOptions</span> <span class="p">{</span> <span class="n">IncludeDeprecationReasons</span> <span class="p">=</span> <span class="k">true</span><span class="p">,</span> <span class="n">IncludeDescriptions</span> <span class="p">=</span> <span class="k">true</span> <span class="p">}))</span> <span class="p">{</span> <span class="n">context</span><span class="p">.</span><span class="n">Response</span><span class="p">.</span><span class="n">ContentType</span> <span class="p">=</span> <span class="s">"application/text"</span><span class="p">;</span> <span class="n">context</span><span class="p">.</span><span class="n">Response</span><span class="p">.</span><span class="n">StatusCode</span> <span class="p">=</span> <span class="p">(</span><span class="kt">int</span><span class="p">)</span> <span class="n">HttpStatusCode</span><span class="p">.</span><span class="n">OK</span><span class="p">;</span> <span class="k">await</span> <span class="n">context</span><span class="p">.</span><span class="n">Response</span><span class="p">.</span><span class="nf">WriteAsync</span><span class="p">(</span><span class="n">printer</span><span class="p">.</span><span class="nf">Print</span><span class="p">());</span> <span class="k">return</span><span class="p">;</span> <span class="p">}</span> </code></pre></div></div> <p>I’m using the code in my GraphQL middleware where if no query was posted then we return the schema.</p> <p>Hope this helps someone.</p> Tue, 26 Feb 2019 00:00:00 +0000 http://compiledexperience.com/blog/posts/exporting-graphql-schema http://compiledexperience.com/blog/posts/exporting-graphql-schema nigel.sampson@compiledexperience.com (Nigel Sampson) csharp graphql EF Core Client Side evaluation (and how to stop it) <p>EF Core has a feature that supports parts of a query being evaluated on the server and parts on the client, the decision is driven by the whether the underlying LINQ provider can convert the expression into SQL.</p> <p>An example would be if we had a method like the following:</p> <div class="language-csharp highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">public</span> <span class="k">static</span> <span class="kt">decimal</span> <span class="nf">CalculateTax</span><span class="p">(</span><span class="kt">decimal</span> <span class="n">price</span><span class="p">)</span> <span class="p">=&gt;</span> <span class="n">price</span> <span class="p">*</span> <span class="m">0.15</span><span class="n">m</span><span class="p">;</span> </code></pre></div></div> <p>and we used it in the query that looked like</p> <div class="language-csharp highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kt">var</span> <span class="n">products</span> <span class="p">=</span> <span class="n">context</span><span class="p">.</span><span class="n">Products</span> <span class="p">.</span><span class="nf">OrderBy</span><span class="p">(</span><span class="n">p</span> <span class="p">=&gt;</span> <span class="n">p</span><span class="p">.</span><span class="n">Price</span><span class="p">)</span> <span class="p">.</span><span class="nf">Select</span><span class="p">(</span><span class="n">p</span> <span class="p">=&gt;</span> <span class="k">new</span> <span class="p">{</span> <span class="n">p</span><span class="p">.</span><span class="n">Id</span><span class="p">,</span> <span class="n">p</span><span class="p">.</span><span class="n">Price</span><span class="p">,</span> <span class="n">Tax</span> <span class="p">=</span> <span class="nf">CalculateTax</span><span class="p">(</span><span class="n">p</span><span class="p">.</span><span class="n">Price</span><span class="p">)</span> <span class="p">});</span> </code></pre></div></div> <p>the LINQ provider knows how to convert the <code class="highlighter-rouge">OrderBy</code> clause to SQL so that will be run on the server, but it has no idea how to convert the <code class="highlighter-rouge">CalculateTax</code> method to SQL so the <code class="highlighter-rouge">Select</code> clause will be run on the client. From the point of view of the developer unless you’re looking very carefully at your query it’s not immediately apparent what will run where.</p> <p>Consider the following query:</p> <div class="language-csharp highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kt">var</span> <span class="n">products</span> <span class="p">=</span> <span class="n">context</span><span class="p">.</span><span class="n">Products</span> <span class="p">.</span><span class="nf">Where</span><span class="p">(</span><span class="n">p</span> <span class="p">=&gt;</span> <span class="nf">CalculateTax</span><span class="p">(</span><span class="n">p</span><span class="p">.</span><span class="n">Price</span><span class="p">)</span> <span class="p">&gt;</span> <span class="m">100.0</span><span class="n">m</span><span class="p">);</span> </code></pre></div></div> <p>Again we can’t evaulate <code class="highlighter-rouge">CalculateTax</code> server side so it’s evalated client side. For this to happen we’ve had to pull the entire contents of the Products table into memory! If this table is of a significant size then the performance problems in terms of memory and time are going to be really nasty.</p> <p>Now both of the above queries are pretty simple, it’s relatively easy to determine what the client / server execution breakdown will look like, however as queries become more complex this task becomes harder and you run the risk of introducing nasty performence regressions.</p> <p>EF Core will log when it drops from server to client evaluation it will log this occurance, but it can be easy to miss.</p> <p>To sum this feature up, I’d avoid it like the plague. Client side evaluation makes it too easy to write a query that has unintended performance regressions without noticing until it’s till late.</p> <p>In my opinion it’s better to be explicit on defining where the query happens. But the best first step is disabling client side evaluation, the following code on the <code class="highlighter-rouge">DbContext</code> changes the drop from server to client from a log warning to an exception.</p> <div class="language-csharp highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">protected</span> <span class="k">override</span> <span class="k">void</span> <span class="nf">OnConfiguring</span><span class="p">(</span><span class="n">DbContextOptionsBuilder</span> <span class="n">optionsBuilder</span><span class="p">)</span> <span class="p">{</span> <span class="n">optionsBuilder</span> <span class="p">.</span><span class="nf">ConfigureWarnings</span><span class="p">(</span><span class="n">w</span> <span class="p">=&gt;</span> <span class="n">w</span><span class="p">.</span><span class="nf">Throw</span><span class="p">(</span><span class="n">RelationalEventId</span><span class="p">.</span><span class="n">QueryClientEvaluationWarning</span><span class="p">));</span> <span class="p">}</span> </code></pre></div></div> <p>This means that whenever you write a query that can’t fully be evaluated in SQL an exception will be thrown and we’ll be fully aware of our problem (at development time).</p> <p>Revisiting our earlier query that would now throw an exception would need to be written to something like in order to not throw that exception.</p> <div class="language-csharp highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kt">var</span> <span class="n">products</span> <span class="p">=</span> <span class="n">context</span><span class="p">.</span><span class="n">Products</span><span class="p">.</span><span class="nf">ToList</span><span class="p">();</span> <span class="n">products</span> <span class="p">=</span> <span class="n">products</span><span class="p">.</span><span class="nf">Where</span><span class="p">(</span><span class="n">p</span> <span class="p">=&gt;</span> <span class="nf">CalculateTax</span><span class="p">(</span><span class="n">p</span><span class="p">.</span><span class="n">Price</span><span class="p">)</span> <span class="p">&gt;</span> <span class="m">100.0</span><span class="n">m</span><span class="p">).</span><span class="nf">ToList</span><span class="p">();</span> </code></pre></div></div> <p>It’s now very clear what is being evaulated server side and what’s on the client side and hopefully the code smell of the entire Products table being loaded into memory is very apparent.</p> <p>In short, disable this feature as the first thing you do in order to not shoot yourself in the foot.</p> <p>Hope this helps.</p> Wed, 30 Jan 2019 00:00:00 +0000 http://compiledexperience.com/blog/posts/ef-core-client-side-eval http://compiledexperience.com/blog/posts/ef-core-client-side-eval nigel.sampson@compiledexperience.com (Nigel Sampson) csharp ef Moving EF Core migrations to their own assembly <p>If you follow an introduction to EF Core tutorial then you’ll tend to find the migrations end up in the same assembly as the models and DB context. For a lot of people this is fine but personally I prefer to move them off to their own assembly. For me this is about seperating a run time concern (the model and the context) from a deployment concern (the migrations), but this is completely subjective.</p> <p>So if you’ve been following the tutorials and have all these concerns in the same assembly what does it take to seperate them? Thankfully not too much.</p> <p>For the sake of this example we’ll assume I have a project <code class="highlighter-rouge">MyMicroService.Core</code> which has a <code class="highlighter-rouge">Migrations</code> folder along side our application code as well as <code class="highlighter-rouge">MyMicroService.API</code> that is the ASP.NET Core project that makes use of the EF Core models.</p> <ol> <li>Create the project <code class="highlighter-rouge">MyMicroService.Core.Migrations</code> to hold the migrations. I deliberately named this to match it up with the existing namespace of the migrations.</li> <li>Add refereces to EF Core and related packages to the migrations project, in my case this included <code class="highlighter-rouge">Npgsql.EntityFrameworkCore.PostgreSQL</code>.</li> <li>Add a reference from <code class="highlighter-rouge">MyMicroService.Core.Migrations</code> to <code class="highlighter-rouge">MyMicroService.Core</code>. The generated migrations reference the <code class="highlighter-rouge">DbContext</code> in code like the following, so the migrations project needs to reference the core project. <div class="language-csharp highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="na">[DbContext(typeof(MyMicroServiceDbContext))]</span> </code></pre></div> </div> </li> <li>Copy the migrations and model snapshot files to the new project. Given the naming we did in step 1 this should just be a copy / paste of the folder in question.</li> <li>Configure the <code class="highlighter-rouge">DbContext</code> with the new assebmly <div class="language-csharp highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">options</span><span class="p">.</span><span class="nf">UseNpgsql</span><span class="p">(</span><span class="n">connectionString</span><span class="p">,</span> <span class="n">o</span> <span class="p">=&gt;</span> <span class="n">o</span><span class="p">.</span><span class="nf">MigrationsAssembly</span><span class="p">(</span><span class="s">"MyMicroService.Core.Migrations"</span><span class="p">));</span> </code></pre></div> </div> </li> <li>Add a reference from the startup assembly <code class="highlighter-rouge">MyMicroService.API</code> to the migrations assembly. This is lets us do things such as check if our database is up to date etc.</li> </ol> <p>And we’re done, hope this helps.</p> Wed, 23 Jan 2019 00:00:00 +0000 http://compiledexperience.com/blog/posts/ef-core-migrations http://compiledexperience.com/blog/posts/ef-core-migrations nigel.sampson@compiledexperience.com (Nigel Sampson) csharp ef Interesting uses of tuple deconstruction <p>C# 7.0 brought us a new and interesting feature with tuple types and tuple literals. These coupled with tuple deconstruction let us create new syntax patterns and helper methods that hopefully results in more readable code.</p> <p>One bug bear of mine was the way we ended up having to use <code class="highlighter-rouge">Task.WhenAll</code> when wanting to await mutiple tasks at the same time. If you didn’t need the results of the <code class="highlighter-rouge">Task</code>’s then it wasn’t too bad.</p> <div class="language-csharp highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">await</span> <span class="n">Task</span><span class="p">.</span><span class="nf">WhenAll</span><span class="p">(</span><span class="nf">StartTaskOne</span><span class="p">(),</span> <span class="nf">StartTaskTwo</span><span class="p">());</span> </code></pre></div></div> <p>but when we want the results of the tasks when end up with syntax that looks like</p> <div class="language-csharp highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kt">var</span> <span class="n">count</span> <span class="p">=</span> <span class="nf">GetCount</span><span class="p">();</span> <span class="kt">var</span> <span class="n">description</span> <span class="p">=</span> <span class="nf">GetDescription</span><span class="p">();</span> <span class="k">await</span> <span class="n">Task</span><span class="p">.</span><span class="nf">WhenAll</span><span class="p">(</span><span class="n">count</span><span class="p">,</span> <span class="n">description</span><span class="p">);</span> <span class="n">Console</span><span class="p">.</span><span class="nf">WriteLine</span><span class="p">(</span><span class="s">$"Count: </span><span class="p">{</span><span class="n">count</span><span class="p">.</span><span class="n">Result</span><span class="p">}</span><span class="s">, Description: </span><span class="p">{</span><span class="n">description</span><span class="p">.</span><span class="n">Result</span><span class="p">}</span><span class="s">"</span><span class="p">);</span> </code></pre></div></div> <p>What we can do is create an extension method that extends a <code class="highlighter-rouge">ValueTuple</code> of <code class="highlighter-rouge">Tasks</code>’s awaits them and results another tuple of the results.</p> <div class="language-csharp highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">public</span> <span class="k">static</span> <span class="k">class</span> <span class="nc">TaskExtensions</span> <span class="p">{</span> <span class="k">public</span> <span class="k">static</span> <span class="k">async</span> <span class="n">Task</span><span class="p">&lt;</span><span class="n">ValueTuple</span><span class="p">&lt;</span><span class="n">T1</span><span class="p">,</span> <span class="n">T2</span><span class="p">&gt;&gt;</span> <span class="n">WhenAll</span><span class="p">&lt;</span><span class="n">T1</span><span class="p">,</span> <span class="n">T2</span><span class="p">&gt;(</span><span class="k">this</span> <span class="n">ValueTuple</span><span class="p">&lt;</span><span class="n">Task</span><span class="p">&lt;</span><span class="n">T1</span><span class="p">&gt;,</span> <span class="n">Task</span><span class="p">&lt;</span><span class="n">T2</span><span class="p">&gt;&gt;</span> <span class="n">tasks</span><span class="p">)</span> <span class="p">{</span> <span class="k">await</span> <span class="n">Task</span><span class="p">.</span><span class="nf">WhenAll</span><span class="p">(</span><span class="n">tasks</span><span class="p">.</span><span class="n">Item1</span><span class="p">,</span> <span class="n">tasks</span><span class="p">.</span><span class="n">Item2</span><span class="p">);</span> <span class="k">return</span> <span class="p">(</span><span class="n">tasks</span><span class="p">.</span><span class="n">Item1</span><span class="p">.</span><span class="n">Result</span><span class="p">,</span> <span class="n">tasks</span><span class="p">.</span><span class="n">Item2</span><span class="p">.</span><span class="n">Result</span><span class="p">);</span> <span class="p">}</span> <span class="p">}</span> </code></pre></div></div> <p>When then end up with more readable</p> <div class="language-csharp highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kt">var</span> <span class="p">(</span><span class="n">count</span><span class="p">,</span> <span class="n">description</span><span class="p">)</span> <span class="p">=</span> <span class="k">await</span> <span class="p">(</span><span class="nf">GetCount</span><span class="p">(),</span> <span class="nf">GetDescription</span><span class="p">()).</span><span class="nf">WhenAll</span><span class="p">();</span> <span class="n">Console</span><span class="p">.</span><span class="nf">WriteLine</span><span class="p">(</span><span class="s">$"Count: </span><span class="p">{</span><span class="n">count</span><span class="p">}</span><span class="s">, Description: </span><span class="p">{</span><span class="n">description</span><span class="p">}</span><span class="s">"</span><span class="p">);</span> </code></pre></div></div> <p>Sadly the above extension method only works with two tasks, but you can see how you’d write a three task version. As the C# language stands right now there’s no way to build a generic version that would work with any amount of tasks (though I’d be very happy to be proven wrong here). I’d argue though if you need overloads of this method beyond four or five something has gone awfully wrong in your code base.</p> Tue, 18 Dec 2018 00:00:00 +0000 http://compiledexperience.com/blog/posts/abusing-tuples http://compiledexperience.com/blog/posts/abusing-tuples nigel.sampson@compiledexperience.com (Nigel Sampson) csharp Project references to multi-targeted projects <p>During the process of moving Caliburn.Micro to .NET Standard and the new <a href="https://caliburnmicro.com/announcements/net-standard">multi-targeting project format</a> I’ve encountered a number of issues in the tooling around intellisense and builds. This isn’t surprising given the relative newness of this approach, but I thought I’d share some of the issues over the next few weeks to help you out.</p> <p>One of the first things I did was move <code class="highlighter-rouge">Caliburn.Micro.Platform</code> from a number of projects (around five I believe) in the same folder (one for each platform) to the new “SDK style” project format which allows multiple outputs based on a series of target frameworks (rather than the normal singular framework).</p> <p>This worked out fine, but the unit tests in the solution started failing with compilation errors where certain classes were missing. In this case it was classes that aren’t present in the Xamarin.Forms platform. This unit test project was a .NET 4.5 project and was clearly picking the wrong output of the multi-targeted project. Instead of picking the “best” platform of .NET 4.5, it was picking the widest in .NET Standard 1.4.</p> <p>This is a known issue and you can see it being discussed on the GitHub repository for the new project system under <a href="https://github.com/dotnet/project-system/issues/1162">“P2P refs choose first tfm in multi-targting reference, not closest one in legacy project system”</a>.</p> <p>If you’ve read the above issue you’ll notice the way to solve this is to shift the other project to the new poject system. Once this is done the new project will pick the correct project output.</p> <p>As I run into more problems during this port (hopefully not too many) I’ll post them up here.</p> Tue, 14 Nov 2017 00:00:00 +0000 http://compiledexperience.com/blog/posts/project-references-multi-targeting http://compiledexperience.com/blog/posts/project-references-multi-targeting nigel.sampson@compiledexperience.com (Nigel Sampson) csharp xamarin