Vitruvion is a transformer-based model trained to generate parametric computer-aided design (CAD) sketches. It shows promise to augment mechanical design via tasks such as converting hand drawings to CAD models, autocompleting sketches, and inferring intended constraints.
Overview
Parametric CAD tools are the predominant way engineers specify physical structures, from bicycle pedals to airplanes to printed circuit boards. They empower users to explore parameterized variations on their designs, while also providing a structured representation for simulation and manufacturing.
At an operational level, parametric CAD starts with the specification of two-dimensional geometric representations, referred to as "sketches". A sketch consists of a collection of geometric primitives (e.g., lines, circles), along with a set of constraints that relate these primitives to one another. Example constraints include parallelism, tangency, and coincidence. This relational specification allows edits to coherently propagate to other parts of the design.
Parametric CAD begins with simple primitives (e.g., lines, circles) (left) that are combined with constraints (e.g., perpendicularity, distance) to form 2D sketches (middle). These sketches then form the geometric foundation for 3D parts (right). Constraints enable "edit propagation", where components of CAD models are updated automatically in response to user-imposed changes. A sketch shown in the program Onshape (left) is updated as its numerical constraint parameters (e.g., radii, angles) are altered (right).
While this paradigm is powerful, it is often a challenging and tedious process. Machine learning offers the intriguing possibility of automatically encoding commonly used patterns, mitigating the inefficiencies involved in repetitive manual design. This could enable new tools such as autocompletion, constraint inference, and conditional synthesis.
In this work, we develop an approach to generative modeling of parametric CAD sketches. Vitruvion, trained on real-world designs from the SketchGraphs dataset, synthesizes CAD sketches by autoregressively sampling geometric primitives, with initial coordinates, and constraints that reference back to the sampled primitives. Because samples from the model match the constraint graph representation used in standard CAD software, they may be imported solved, and edited according to downstream design tasks. In addition, we experiment with providing various kinds of context to the model, including partial sketches (primers) and images of hand-drawn sketches, enabling conditional synthesis.
We factorize CAD sketch synthesis into two sequence modeling tasks: primitive generation (left) and constraint generation (right). The primitive model may be conditioned on an optional input context, such as a hand-drawn sketch.
Example results
Extracting primitives from hand-drawn sketches
Raster images of hand-drawn sketches (left) are provided as input to the primitive model, which then generates sequences of primitives (right). For each input here, four samples are generated.
Autoconstraining
Unconstrained geometry (left) is passed to the constraint model and then solved (right). Images are screenshots from the Onshape user interface. An example of conditional reasoning in the constraint model. In the first case (top), the model constrains the top horizontal line to serve as the top of the rectangle (coincident with the circles' centers). In the second case (bottom), we manually add another line segment (denoted by the green arrow) just below the top one. Now, the model constrains this new line to be the top of the rectangle, setting the original one to be coincident with the circumferences of the two top circles.
Editing generated sketches
Hand-drawn sketches (left) are converted to initial unsolved sketches (middle) and then imported to Onshape, solved, and edited to various configurations (right). Constraints help enforce design intent while allowing edits to propagate.
Autocompleting
We take held-out sketches (leftmost column) and delete 40% of their primitives to form primers (second column from left) for the primitive model. To the right are the inferred completions, where the model is queried for additional primitives until selecting a stop token.
Why "Vitruvion"?
While debugging a data augmentation procedure, we happened to render the average of a set of normalized SketchGraphs sketches. In our opinion, it was the spitting image of a famous 1490 drawing by Leonardo da Vinci, The Vitruvian Man, depicting a man inscribed in a circle and square. Likewise, we decided to name our model after this drawing (albeit with a slight spelling alteration).
Heatmap resulting from average normalized SketchGraphs images (left) and "The Vitruvian Man" by Leonardo Da Vinci (right). The two images are almost indistinguishable, hence our naming choice.
SARS-CoV-2 cases are beginning to rise again as we head into October. In part due to Labor Day celebrations and the return of many students to in-person classes, the latest National CDC forecasts predict up to 7,400 new COVID-19 deaths per week in our near future. Back during the first spike, one of the major challenges that emerged was a shortage of personal protective equipment and resulting threat to our medical practitioners. Many academic communities paused research pursuits to address this challenge, with my group among them. This post covers our successes, setbacks, and a vision for addressing inevitable future shortages through effective, distributed rapid-response manufacturing.
In early April 2020, after daily COVID-19 infections in New York had increased from 29 to 10,480 in a single month, medical professionals faced a tough road ahead. Lombardy, Italy appeared to present a worst-case study of the near future, with doctors and nurses falling ill due to lack of protective personal equipment (PPE) and ventilator shortages threatening the lives of patients. New York hospitals were struggling to secure protection for their staff as sudden demand and consumer panic-buying dried up traditional supply chains. A short drive away in Princeton, my lab, led by Prof. Ryan Adams, held a team meeting to think through how we could help. I decided to pause my research to lead a project that converted our lab's prototyping equipment into a COVID face shield manufacturing pipeline.
The Process
Our lab received an incoming request from a hospital in NY in need of face shields in late March. Within 48 hours, I was able to bring our 3D printers into full working order, then build and prototype face shields based on available online designs. Throughout this process I considered various trade-offs, such as throwaway vs. re-use, strength vs. disposability, safety of materials, and speed of manufacture. I settled on a design by Prusa3D, certified by a European health ministry.
N95-style
Prusa-style
Complete prototype
Rapid prototyping for manufacturing speed, materials, and part quality. Right: chosen design.
While my colleague Xingyuan secured donations for sterile face coverings and gloves, I set about developing a manufacturing plan for my research group. Everyone was eager to help, but we needed to maintain social distancing protocols to keep each other safe. After rearranging our lab space, I was able to coordinate 5 volunteers working morning and night shifts (with a maximum of 2 persons in the lab) to keep things running.
Xingyuan
Yaniv
Alex
Jordan
Me
Volunteers from the lab. Not pictured: Sulin and Daniel, who also contributed.
We made our initial shipment three days after finalizing the design, a second shipment three days later. We continued to ship two boxes a week for the next month, until we determined that slower-moving but higher volume manufacturing of face shields had finally caught up with the need. In the height of that first spike we churned out 500 shields through a process that reliably output 150 completed parts per week.
Milstein Hospital after receiving our first shipment
Key Challenges
It was difficult to determine the minimal acceptable product for hospitals under pandemic conditions. A face shield that does nothing is worse than nothing at all because of the false sense of protection it gives the wearer. In response to this, we communicated directly with medics and iterated based on their feedback.
Health and safety was another key concern: this all took place during a pandemic and social shutdown. We ourselves needed basic PPE, cleaning supplies, and masks. The Association of Chinese Students and Scholars at Princeton University (ACSSPU) generously donated non-medical face masks, and I was able to purchase disinfecting supplies at the then largely-unutilized university student store.
"Stringing" print failure
On the manufacturing side, we were printing at max capacity of our 3D printer beds. Tuning the parameters of the print (such as plastic extrusion speed, tool head travel speed, and temperature) was crucial and difficult. Small environmental changes sometimes affected the 3D prints and ruined entire batches.
Cracked motor bracket
Maintenance of these hobbyist machines being run on a 24-hour cycle was also an issue. We had small windows in which to do maintenance in order to maximize output. Initially, I vacillated between repair and maintenance and training my colleagues in manufacturing. But after a week of constant troubleshooting, I invested time in writing out both printing and maintenance steps, and did more in-person maintenance training. This paid off as the volunteers got better and better at diagnosing and fixing problems, and our cycle time improved.
Finally, just as traditional manufacturers have bottlenecks, our process did as well. 3D printing materials began to sell out everywhere by the end of April, so moving fast to secure raw materials and talking directly to suppliers was key.
Lessons Learned.
It’s crucial to communicate progress in order to build momentum and keep the team motivated during a time of global crisis. Everyone was grappling with their own fears for safety. I worked to highlight the growing output of potentially life-saving equipment built by our team each week, and worked to ensure everyone’s safety was kept paramount.
Developing a process to maximize face shield output using machines and labour power is easy on paper, and hard in practice. I seriously underestimated the time required for initial parameter-tuning of 3D printers, and for ongoing maintenance and cleaning to keep them running nonstop.
Broader Implications
There is a problem right now in how we respond to urgent needs in our supply chains.
When demand for a particular item goes from 1x to 1000x in a short period of time, and that item fills an urgent, time-sensitive need, existing economies of scale are unable to respond. This resulted in doctors and medics being unable to get critical protective equipment. Conversely producing everything locally has its own challenges: siloed design iteration, scarcity of key raw materials, and unideal equipment that must be repurposed.
To address this unmet need in the future, I recommend two key preventative approaches:
Build emergency rapid manufacturing responses into general plans for pandemics and other natural disasters, identifying raw materials that may be in highest demand. University engineering and science labs in particular are storehouses of rapid prototyping and manufacturing equipment. All that is required to convert them into emergency-response manufacturing for life-saving equipment is central leadership and organizational acumen.
Fund research into automating 3D manufacturing pain points, like parameter-tuning. 3D printers use Computer Numerical Control (CNC) systems that currently require human intervention for careful parameter tuning.Many off-the-shelf robust and reliable methods in machine learning have not yet been applied here. This hinders amateur volunteers from using their equipment to help. Changing this through innovation presents both an economic opportunity and a humanitarian one. With easier access to automated systems, we can collectively respond better to such crises in the future.
Convex objective functions are the ones we understand the best. This video explains how things like linear programming can capture real-world optimization problems. This is part of a series of videos for COS 302: Mathematics for Numerical Computation and Machine Learning, replacing lectures after the course went remote due to the COVID-19 pandemic.
Optimization is a huge topic of tremendous importance. In this video we hit some of the high points to give you the big picture. This is part of a series of videos for COS 302: Mathematics for Numerical Computation and Machine Learning, replacing lectures after the course went remote due to the COVID-19 pandemic.
Information theory is a fascinating topic that informs many of the ways we think about structure in data. This video provides a brief overview of some of the concepts, inspired by the lectures of my PhD advisor, the late David MacKay. This is part of a series of videos for COS 302: Mathematics for Numerical Computation and Machine Learning, replacing lectures after the course went remote due to the COVID-19 pandemic.
It's difficult to overstate the centrality of the Gaussian distribution to machine learning, statistics, and many other natural sciences. This video talks about some of the reasons this distribution is special. This is part of a series of videos for COS 302: Mathematics for Numerical Computation and Machine Learning, replacing lectures after the course went remote due to the COVID-19 pandemic.
To do interesting things with joint distributions, they need to have dependence structure. This is part of a series of videos for COS 302: Mathematics for Numerical Computation and Machine Learning, replacing lectures after the course went remote due to the COVID-19 pandemic.
From a modeling point of view, joint probability distributions are where the action is. They let you posit latent structure and reason about conditional probability. This video gives an overview of the basics of joint random variables. This is part of a series of videos for COS 302: Mathematics for Numerical Computation and Machine Learning, replacing lectures after the course went remote due to the COVID-19 pandemic.
There are some distributions that come up over and over again in machine learning and statistics. In this video, I give an overview of some distributions to be aware of. This is part of a series of videos for COS 302: Mathematics for Numerical Computation and Machine Learning, replacing lectures after the course went remote due to the COVID-19 pandemic.
