I am a developer. For the past 3 years I have been writing code for the web, mostly in Javascript.
I recently joined Microsoft as an intern to work on the AI Factory team, part of Bing. My project is to build a tool that converts sketches to code. I call it sketch2code.
As an intern, I will be working full time at Microsoft for 8 weeks (June 19th – August 11th). This blog is a way to keep everyone updated on my progress and hopefully encourage a dialog around my project.
If you want to chat with me or learn more about my project, feel free to reach out: @pablobm_ on Twitter or by email [pablo@bing.com](mailto:pablo@bing.com)
sketch2code is an experimental tool that takes hand-drawn design and generates the corresponding HTML code. The tool is currently available as a Visual Studio Code extension. It uses Azure Custom Vision service to infer handwritten text and parse it into HTML elements. The generated markup can be copied to clipboard, previewed in browser or exported to a file.
The blog covers the development of sketch2code as well as related research and tutorials.
Sketch2Code is a tool that converts a hand-drawn HTML design into actual HTML code. It uses Computer Vision to understand the HTML structure of the design and then uses this to map the design to Bootstrap-like markup.
The best way to understand Sketch2Code is by watching the video.
This blog follows the development of Sketch2Code as we work on it over time.
Sketch2Code is a new and exciting tool for developers. And it’s just as exciting for business users too. Sketch2Code is the tool that will allow end users to get involved in the process of building powerful cloud services from their ideas, sketched on paper.
Sketch2Code was developed by Microsoft Research and will soon be available as an open source project for anyone to use, learn about and contribute to.
Why sketch2code?
The idea of turning a hand-drawn mockup into html and CSS is intriguing. It would enable anyone with a computer to create UI for their apps, without having to learn how to code. With the increase in demand for AI/ML experts and data scientists in the market, I believe there will be a growing number of non-technical people who want to create simple User Interface for their projects.
I started thinking about this idea in summer 2018 when my friend and I were working on our first startup. We wanted to build an app that could help users find similar books to the ones they like (or dislike). I was responsible for building the frontend of the app. So we sketched out the wireframes using pen and paper, then I recreated them using HTML and CSS. This is when I had the idea: what if there was a tool that could take our sketches as input, and generate html and css for us?
I started writing about the internal workings of sketch2code here.
The first posts were about how we made the decision to create it, then came the plan and how we built it.
There’s also a demo video, and how we were able to make a prototype so fast using Microsoft Cognitive Services.
I’m now writing a series on usability testing with real users and heuristics evaluations.
Sketch2Code is a project I started about a year ago. The idea was to generate HTML and CSS from hand-drawn wireframes. I received a lot of positive feedback and wanted to continue working on the project, so Microsoft’s AI for Accessibility program was an opportunity I couldn’t pass up.
Thanks to the program, I’ve been able to work on Sketch2Code full-time for the last six months. It’s been an interesting journey, and in this first post, I’ll share some of what I’ve learned.