Vitruvion: A Generative Model of Parametric CAD Sketches

Ari Seff · September 25, 2021

Blog

Ari Seff, Wenda Zhou, Nick Richardson, and Ryan P. Adams
International Conference on Learning Representations (ICLR) 2022
Paper
Video
Code & model weights

Vitruvion is a transformer-based model trained to generate parametric computer-aided design (CAD) sketches. It shows promise to augment mechanical design via tasks such as converting hand drawings to CAD models, autocompleting sketches, and inferring intended constraints.

Overview

Parametric CAD tools are the predominant way engineers specify physical structures, from bicycle pedals to airplanes to printed circuit boards. They empower users to explore parameterized variations on their designs, while also providing a structured representation for simulation and manufacturing.

At an operational level, parametric CAD starts with the specification of two-dimensional geometric representations, referred to as "sketches". A sketch consists of a collection of geometric primitives (e.g., lines, circles), along with a set of constraints that relate these primitives to one another. Example constraints include parallelism, tangency, and coincidence. This relational specification allows edits to coherently propagate to other parts of the design.

Parametric CAD begins with simple primitives (e.g., lines, circles) (left) that are combined with constraints (e.g., perpendicularity, distance) to form 2D sketches (middle). These sketches then form the geometric foundation for 3D parts (right).

Constraints enable "edit propagation", where components of CAD models are updated automatically in response to user-imposed changes. A sketch shown in the program Onshape (left) is updated as its numerical constraint parameters (e.g., radii, angles) are altered (right).

While this paradigm is powerful, it is often a challenging and tedious process. Machine learning offers the intriguing possibility of automatically encoding commonly used patterns, mitigating the inefficiencies involved in repetitive manual design. This could enable new tools such as autocompletion, constraint inference, and conditional synthesis.

In this work, we develop an approach to generative modeling of parametric CAD sketches. Vitruvion, trained on real-world designs from the SketchGraphs dataset, synthesizes CAD sketches by autoregressively sampling geometric primitives, with initial coordinates, and constraints that reference back to the sampled primitives. Because samples from the model match the constraint graph representation used in standard CAD software, they may be imported solved, and edited according to downstream design tasks. In addition, we experiment with providing various kinds of context to the model, including partial sketches (primers) and images of hand-drawn sketches, enabling conditional synthesis.

We factorize CAD sketch synthesis into two sequence modeling tasks: primitive generation (left) and constraint generation (right). The primitive model may be conditioned on an optional input context, such as a hand-drawn sketch.

Example results

Extracting primitives from hand-drawn sketches

Raster images of hand-drawn sketches (left) are provided as input to the primitive model, which then generates sequences of primitives (right). For each input here, four samples are generated.

Autoconstraining

Unconstrained geometry (left) is passed to the constraint model and then solved (right). Images are screenshots from the Onshape user interface.

An example of conditional reasoning in the constraint model. In the first case (top), the model constrains the top horizontal line to serve as the top of the rectangle (coincident with the circles' centers). In the second case (bottom), we manually add another line segment (denoted by the green arrow) just below the top one. Now, the model constrains this new line to be the top of the rectangle, setting the original one to be coincident with the circumferences of the two top circles.

Editing generated sketches

Hand-drawn sketches (left) are converted to initial unsolved sketches (middle) and then imported to Onshape, solved, and edited to various configurations (right). Constraints help enforce design intent while allowing edits to propagate.

Autocompleting

We take held-out sketches (leftmost column) and delete 40% of their primitives to form primers (second column from left) for the primitive model. To the right are the inferred completions, where the model is queried for additional primitives until selecting a stop token.

Why "Vitruvion"?

While debugging a data augmentation procedure, we happened to render the average of a set of normalized SketchGraphs sketches. In our opinion, it was the spitting image of a famous 1490 drawing by Leonardo da Vinci, The Vitruvian Man, depicting a man inscribed in a circle and square. Likewise, we decided to name our model after this drawing (albeit with a slight spelling alteration).

Heatmap resulting from average normalized SketchGraphs images (left) and "The Vitruvian Man" by Leonardo Da Vinci (right). The two images are almost indistinguishable, hence our naming choice.

Share: Twitter, Facebook