Preheat oven to 400F. Using a rolling pin and flour for dusting, roll out puff pastry sheet to 12" x 16" or roughly the size of a baking sheet. Transfer to parchment lined baking sheet. Using a fork, poke holes all over pastry. Baked for 10 minutes until just beginning to lightly brown around edges. Allow to cool slightly.
Meanwhile, bring a large pot of water to a boil. Submerse asparagus for 1-2 minutes. Transfer asparagus to a large bowl of ice water immediately and allow to cool. Transfer to clean towel and dry any excess moisture. Toss asparagus with olive oil and a pinch of salt and pepper.
Combine cheese, egg yolks, milk, shallot, nutmeg and a pinch of salt and pepper in a medium bowl.
Spread cheese mixture over pastry. Top with asparagus, alternating the direction of the heads.
Bake for an additional 12-14 minutes until pastry is golden brown and cheese mixture puffs up from between the stalks of asparagus.
Sprinkle with lemon zest and serve warm or at room temperature.