That would be very difficult to work with. You don’t really want to work with painting tools on code (to erase things for example) and interpretation of the drawing would likely be unstable and change between versions of the glorified OCR tool aka programming language.
Also diffs, version control and debugging would be very hard to get working properly.
If you want to try a sort-of graphical programming language, I suggest downloading MPS: MPS: The Domain-Specific Language Creator by JetBrains, it’s not on the level of using pen input, but you can insert tables, vectors and stuff directly into code (into standard Java code if you like) and you will quickly see the drawbacks of not being able to edit and copy-paste text freely. It feels more like using the Word Equation Editor and less like programming.