018_quantifiedcode.vtt

WEBVTT

00:00:00.001 --> 00:00:03.660
Often, the most important lessons we learn are what not to do.

00:00:03.660 --> 00:00:09.520
Today, we'll be talking about bad Python code and Python anti-patterns with Andreas Duvez.

00:00:09.520 --> 00:00:13.600
It's show number 18, recorded Thursday, July 1st, 2015.

00:00:13.600 --> 00:00:43.580
Welcome to Talk Python To Me, a weekly podcast on Python, the language, the library,

00:00:43.760 --> 00:00:45.260
the ecosystem, and the personalities.

00:00:45.260 --> 00:00:46.880
This is your host, Michael Kennedy.

00:00:46.880 --> 00:00:49.000
Follow me on Twitter, where I'm @mkennedy.

00:00:49.000 --> 00:00:53.100
Keep up with the show and listen to past episodes at talkpythontome.com,

00:00:53.100 --> 00:00:55.660
and follow us on Twitter, where we're at Talk Python.

00:00:55.660 --> 00:01:02.220
This episode, we'll be talking to Andreas Duvez from Quantified Code about Python anti-patterns.

00:01:02.220 --> 00:01:06.660
This episode is brought to you by CodeShip and Hired.

00:01:06.660 --> 00:01:13.060
Thank them for supporting the show on Twitter via at CodeShip and at Hired underscore HQ.

00:01:13.560 --> 00:01:19.020
I don't have any major news for you this week, so let me introduce Andreas so we can get right to the show.

00:01:19.020 --> 00:01:23.840
Andreas is a physicist and software developer with a passion for code quality.

00:01:23.840 --> 00:01:27.600
He leads the development of algorithms and software at Quantified Code.

00:01:27.600 --> 00:01:29.760
Andreas, welcome to the show.

00:01:29.760 --> 00:01:30.880
Thanks.

00:01:30.880 --> 00:01:31.980
Thanks for having me here.

00:01:32.480 --> 00:01:33.780
It's great to have you here.

00:01:33.780 --> 00:01:40.880
We're going to talk about design patterns, both good design patterns and bad design patterns or anti-patterns,

00:01:40.880 --> 00:01:43.580
with the stuff you guys have going on at Quantified Code.

00:01:43.580 --> 00:01:44.820
Exactly.

00:01:44.820 --> 00:01:50.700
So you've built a bunch of awesome tools, both visual and static analysis, and we're going to talk about that.

00:01:50.700 --> 00:01:55.060
But before we get into it, let's just take a step back down memory lane and, you know,

00:01:55.060 --> 00:01:58.460
tell everyone how you got started in Python and programming and what's your story?

00:01:58.500 --> 00:01:59.020
How did you get here?

00:01:59.020 --> 00:02:03.640
So my background is actually not in software development, but in physics.

00:02:04.460 --> 00:02:10.360
And I made my first contact with Python while I was doing my PhD, which was about, like,

00:02:10.360 --> 00:02:12.520
experimental quantum computing.

00:02:12.520 --> 00:02:19.420
So we did experiments with microwaves and lots of signal processing.

00:02:19.420 --> 00:02:25.880
And when I arrived at our lab, we had, like, a lot of equipment that we needed to, like, control

00:02:25.880 --> 00:02:31.520
using programs, and we had a lot of data that we needed to analyze, to visualize, and to process.

00:02:32.160 --> 00:02:38.320
And we did that using various programming languages, for example, Matlab and C and C++.

00:02:38.320 --> 00:02:46.720
And we were also using tools like LabVIEW that help you to, like, define interfaces, graphical user

00:02:46.720 --> 00:02:50.080
things, and, like, control your measurement equipment.

00:02:50.080 --> 00:02:57.580
And then I remember when I came to the lab, this was basically a huge mess because we used, like,

00:02:57.580 --> 00:03:04.540
four different programming languages and the graphical programming interface of LabVIEW, which is nice in principle,

00:03:04.540 --> 00:03:09.260
but which makes it really, really difficult to write complex software systems with that.

00:03:09.600 --> 00:03:13.080
And so I said, hey, there has to be a better way to do this.

00:03:13.080 --> 00:03:19.960
And one of our postdocs, he used Python in his job in the U.S., and he introduced me to it.

00:03:19.960 --> 00:03:26.460
And so I started, like, getting interested in that, and I tried to, like, replace more and more of our software stack with Python.

00:03:26.960 --> 00:03:37.060
And after about one year or half a year, I think, we had replaced most of the data acquisition and data analysis things we had in the lab with Python programs.

00:03:37.480 --> 00:03:41.120
So that was kind of, like, the first contact with me, for me, with that language.

00:03:41.120 --> 00:03:48.200
And also when I, like, that I saw that it's really powerful and you can basically do anything with it.

00:03:48.200 --> 00:03:49.640
And, yeah.

00:03:49.640 --> 00:03:51.060
Yeah, that's great.

00:03:51.060 --> 00:03:55.620
I think a lot of people come to Python for the data science component of it.

00:03:55.620 --> 00:04:06.980
And I think, like you were saying, the fact that it's a full stack, legitimate, end-to-end programming language means you don't have to just live within those little shells of what MATLAB will do or LabVIEW will do.

00:04:07.060 --> 00:04:08.440
But you can bring it all together.

00:04:08.440 --> 00:04:09.320
Yes, exactly.

00:04:09.320 --> 00:04:10.100
Yeah.

00:04:10.100 --> 00:04:15.040
And, I mean, back in 2009, when I started using it, the tooling was already pretty good.

00:04:15.040 --> 00:04:23.700
But if you have a look at it now in 2015, it's really amazing what kind of an ecosystem people have created.

00:04:23.700 --> 00:04:32.620
You know, for example, the IPython notebook, NumPy, SciPy, and all these other tools that we can use to, as you say, do almost anything that we can imagine.

00:04:32.620 --> 00:04:35.280
So it's really, really amazing to see.

00:04:35.280 --> 00:04:36.760
Yeah, it is really amazing.

00:04:36.880 --> 00:04:38.400
And it's great.

00:04:38.400 --> 00:04:42.480
And I think the thing that, among other things, that surprises me is it's free.

00:04:42.480 --> 00:04:47.940
You look at MATLAB and Mathematica, you pay tons of money to be locked into these, like, somewhat confusing.

00:04:47.940 --> 00:04:48.460
Yeah, exactly.

00:04:48.460 --> 00:04:50.900
And it's the power of open source.

00:04:50.900 --> 00:04:52.640
So I think, yeah.

00:04:52.980 --> 00:04:57.300
You're doing all this cool programming and getting into Python for your work at Quantified Code.

00:04:57.300 --> 00:04:58.860
This is a company you co-founded.

00:04:58.860 --> 00:04:59.880
Who do you co-found it with?

00:04:59.880 --> 00:05:00.900
And what do you guys do there?

00:05:01.700 --> 00:05:04.360
So I co-founded it with an old friend of mine.

00:05:04.360 --> 00:05:09.080
And we're doing static code analysis for Python, but also other programming languages.

00:05:09.800 --> 00:05:18.860
And the reason we started this company was that I saw during my PhD that, hey, it's really hard to write good software because I didn't have a background in software development.

00:05:18.860 --> 00:05:21.440
So I wrote lots of very, very bad code.

00:05:21.640 --> 00:05:23.260
And I made a lot of mistakes.

00:05:23.260 --> 00:05:29.140
And I thought it would be really nice to have, like, a system, an automated system that would tell me how to write good code.

00:05:29.140 --> 00:05:32.720
So that's when I got interested in all these things like static code analysis.

00:05:32.720 --> 00:05:37.800
And I started looking into what kind of tools existed for Python and other languages.

00:05:37.800 --> 00:05:43.340
And I found that most of the tools that we're using today are still not good enough or they could be much better.

00:05:43.480 --> 00:05:53.240
And so that's kind of our mission to, like, make these tools much better and allow anyone to, like, have access to really high-quality, state-of-the-art code analysis.

00:05:53.240 --> 00:05:54.780
Yeah, that's great.

00:05:54.780 --> 00:05:59.820
One of the problems with a lot of static code analysis is it's very good at finding the small rules.

00:05:59.820 --> 00:06:03.020
Like, hey, you should have white space between these things.

00:06:03.020 --> 00:06:04.860
You should have a new line here.

00:06:04.860 --> 00:06:06.320
Your line shouldn't be this long.

00:06:06.320 --> 00:06:07.700
Your variables should be named this way.

00:06:07.700 --> 00:06:08.180
Yeah, exactly.

00:06:08.180 --> 00:06:10.860
But that's, like, the stuff that robots can do, you know?

00:06:10.860 --> 00:06:17.940
The stuff that's really valuable is the hard-learned lessons of if you write your code this way, it's easier to maintain.

00:06:17.940 --> 00:06:22.300
If you write your code that way, it will actually run faster or it will be less error-prone.

00:06:22.300 --> 00:06:28.780
And that's normally what you get by working with kind of a mentoring apprenticeship, you know, working in a team.

00:06:28.780 --> 00:06:32.840
But it's also the whole concept of design patterns, right?

00:06:32.840 --> 00:06:34.340
Yes, exactly.

00:06:34.340 --> 00:06:43.680
I mean, design patterns certainly help people to become productive and to use solutions that are better proven, so to say, and tested in the field.

00:06:43.680 --> 00:06:50.420
I mean, a design pattern in that sense is just like a reusable solution that you can take and implement in your own code.

00:06:51.280 --> 00:06:59.760
And, I mean, I wasn't even aware of that, but I also made use of design patterns, which are covered, like, from the internet for the work I did during my PhD.

00:06:59.760 --> 00:07:13.080
I used, for example, the observer pattern and the command pattern and other things, which I just, like, yeah, intuitively could adapt to my own needs and use in my projects.

00:07:13.260 --> 00:07:27.000
And the great thing about this is also that it gives you, like, a common language to talk to other people, you know, because if you say, hey, I got this class, which is, like, receiving a notification from this other class and then I was doing some things.

00:07:27.000 --> 00:07:33.520
I mean, it's okay, but you could just say, hey, I'm using this observer class and then everyone kind of knows what you mean, you know?

00:07:33.740 --> 00:07:39.240
Yeah, I think that is one of the absolute really strong powers of design patterns.

00:07:39.240 --> 00:07:46.380
Obviously, one is to know the solutions, but the other is to have larger building blocks as you think about your application mentally.

00:07:46.380 --> 00:07:47.100
Yes, exactly.

00:07:47.100 --> 00:07:47.820
All right.

00:07:47.820 --> 00:07:53.900
Like you say, if I go to you and I describe, okay, what I'm going to do is I'm going to have a variable and the variable is going to be shared.

00:07:53.900 --> 00:07:57.920
There's only going to be one of them and it's going to be shared across all parts of the application.

00:07:57.920 --> 00:08:05.260
And so when one part changes it, the other part will see that same change, I could have this conversation or I could just say I'm using the singleton pattern.

00:08:05.260 --> 00:08:06.240
Yes, exactly.

00:08:06.240 --> 00:08:07.780
You know what I'm talking about.

00:08:07.780 --> 00:08:09.600
You know what the positive side of that is.

00:08:09.600 --> 00:08:11.020
You know what the negative side of that is.

00:08:11.020 --> 00:08:14.180
You know singleton pattern is a challenge for unit testing.

00:08:14.180 --> 00:08:16.480
And so is that a tradeoff you want to make?

00:08:16.480 --> 00:08:19.340
And it's not just helpful for working in a team.

00:08:19.340 --> 00:08:20.300
I can tell you that.

00:08:20.300 --> 00:08:21.440
You could tell me that.

00:08:21.440 --> 00:08:26.120
But I think it's also helpful just for you internally as you think about how your code is structured.

00:08:26.120 --> 00:08:27.720
Yeah, absolutely.

00:08:27.720 --> 00:08:32.580
We've all written a lot of code and we may or may have not given it good attention.

00:08:32.580 --> 00:08:34.880
We may have written it bad.

00:08:34.880 --> 00:08:36.080
We may have written it well.

00:08:36.080 --> 00:08:37.180
We might have been in a hurry.

00:08:37.180 --> 00:08:41.540
And so, or, you know, much more common, we've gotten it from somebody else.

00:08:41.540 --> 00:08:43.580
And we have to deal with it.

00:08:43.580 --> 00:08:50.520
And I think understanding these design patterns is really interesting, but also understanding when they're needed.

00:08:50.940 --> 00:08:55.780
And so there's this related idea of this concept called code smells.

00:08:55.780 --> 00:08:59.080
I think it was introduced by Martin Fowler, but I'm not entirely sure.

00:08:59.080 --> 00:08:59.680
Are you familiar with it?

00:08:59.680 --> 00:09:00.340
Yes, it's possible.

00:09:00.340 --> 00:09:00.600
Yeah.

00:09:01.100 --> 00:09:02.600
I'm not 100% sure.

00:09:02.600 --> 00:09:09.920
I mean, design patterns has been introduced by this gang of four in 1994, but I'm not sure where the term code smells comes from.

00:09:10.440 --> 00:09:12.800
Yeah, certainly Martin Fowler wrote a lot about it.

00:09:12.800 --> 00:09:16.580
And I think it's a very interesting way to think of things.

00:09:16.580 --> 00:09:20.880
So I'll put a link to some code smells in the show notes.

00:09:20.880 --> 00:09:24.940
And I think kind of what your code does is it sort of looks for code smells.

00:09:24.940 --> 00:09:30.380
It says you're doing these things that are not necessarily going to make your code not run, but it's not the best way to do it.

00:09:30.380 --> 00:09:31.660
Yes, exactly.

00:09:31.660 --> 00:09:43.120
I mean, what a static code analyzer does is basically that it looks for patterns in the code that can either produce bugs or are quality problems of the code.

00:09:43.420 --> 00:09:49.680
And so for us, like code smell can be categorized by how severe it is.

00:09:49.680 --> 00:09:57.300
So it might make your code crash or it might just be an annoyance to another programmer who takes longer to understand the code.

00:09:57.300 --> 00:10:01.740
And also by various categories like does it affect the performance?

00:10:01.740 --> 00:10:03.900
Does it affect the security of the code?

00:10:03.900 --> 00:10:05.400
Does it affect the maintainability?

00:10:05.400 --> 00:10:09.900
So there are lots of lots of different types of code smells in that sense.

00:10:10.120 --> 00:10:18.900
Yeah, and if you think back, you know, just thinking of the code smells that I can remember, there's some good ones like duplicated code.

00:10:18.900 --> 00:10:20.060
Oh, yeah.

00:10:20.060 --> 00:10:22.100
That one's common, right?

00:10:22.100 --> 00:10:23.080
Yes.

00:10:23.080 --> 00:10:27.960
Or feature envy or, you know, long method.

00:10:27.960 --> 00:10:31.260
Or there's a bunch of, you know, too many parameters.

00:10:32.260 --> 00:10:32.580
Yes.

00:10:32.580 --> 00:10:35.420
I mean, this we're seeing a lot in the code we analyze.

00:10:35.420 --> 00:10:43.360
Very complex functions, a lot of parameters that have too many responsibilities and are really hard to understand.

00:10:43.360 --> 00:10:50.840
So, I mean, I think that's really the number one performance or maintainability killer today in software complexity.

00:10:51.120 --> 00:10:51.460
I agree.

00:10:51.460 --> 00:10:57.400
One of the really interesting code smells, I think, are comments.

00:10:57.400 --> 00:11:00.880
And so you think, you know, comments, I'm supposed to comment my code.

00:11:00.880 --> 00:11:01.660
That's a good thing.

00:11:01.660 --> 00:11:06.880
A lot of times programmers will use comments where they should have just written better code.

00:11:06.880 --> 00:11:07.960
Yes, exactly.

00:11:07.960 --> 00:11:11.740
They'll write a function that has a bad name and it's not well written.

00:11:11.740 --> 00:11:12.680
So it's hard to understand.

00:11:12.680 --> 00:11:27.860
And it'll put a nice little comment to describe what it does because it's not well written, you know, where the real fix would be to write a well, to refactor, to have a proper descriptive name and to be cleaner in smaller parts, right?

00:11:27.860 --> 00:11:28.700
Yeah, absolutely.

00:11:28.700 --> 00:11:29.320
Yeah.

00:11:29.320 --> 00:11:32.540
Comments are kind of like deodorant for code smells, right?

00:11:32.540 --> 00:11:34.280
That's a good conversion, yeah.

00:11:35.100 --> 00:11:45.160
Okay, so that brings us to how I kind of got connected to you guys or learned about you guys is through this thing you wrote called The Little Book of Python Anti-Patterns.

00:11:45.160 --> 00:11:46.120
Yes.

00:11:46.120 --> 00:11:47.400
Can you tell us about that?

00:11:47.400 --> 00:11:48.620
Yeah, sure.

00:11:48.620 --> 00:11:57.020
So this was a project we started because we wanted to get an overview of what problems you would find in Python code.

00:11:57.020 --> 00:12:11.460
And so we started collecting problem types and code smells and issues from different sources, for example, lintes like PyLint or PyFlakes that already have like a collection of a few hundred of those anti-patterns.

00:12:12.080 --> 00:12:20.620
But also from sources such as Stack Overflow and articles, blog posts, and also from people and from code that we were reading.

00:12:20.620 --> 00:12:28.760
And so we like collected all this together and then we worked with a freelancer in the beginning who helped us like write a large part of the initial text.

00:12:28.760 --> 00:12:40.420
And we bundled this whole collection of anti-patterns together into a book so that people could just like learn about different things they should avoid in their Python code.

00:12:40.420 --> 00:12:44.800
So it's not like learning from good code, but learning from bad code in this sense.

00:12:44.800 --> 00:12:45.660
Yeah.

00:12:45.660 --> 00:12:47.980
You know, it's related to that code smells idea.

00:12:47.980 --> 00:12:58.760
Like I see all these problems and if I know to avoid them, well, either you know the right design pattern and the right style or you just know you need to go do some research and don't do it this way.

00:12:58.760 --> 00:13:00.060
Yeah, exactly.

00:13:00.060 --> 00:13:00.400
Yeah.

00:13:00.400 --> 00:13:00.840
Yeah.

00:13:00.840 --> 00:13:06.100
In your book, you do have sort of recommendations for most of the anti-patterns, right?

00:13:06.100 --> 00:13:07.320
Yes.

00:13:07.840 --> 00:13:13.720
So we always, if we describe the problem, we always try to like give at least one or two solutions to it.

00:13:13.720 --> 00:13:24.060
I mean, sometimes there is no really unique solutions or people would kind of like disagree if it is a problem at all or not.

00:13:24.060 --> 00:13:31.040
So, but we are always pretty opinionated and we think like a good programmer should be a bit opinionated.

00:13:31.220 --> 00:13:35.900
And so we try to like give our solution that we would actually solve the problem with.

00:13:35.900 --> 00:13:37.140
Yeah, that's great.

00:13:37.140 --> 00:13:44.140
And let me just tell everybody that the book is at docs.quantifiedcode.com slash python dash code dash patterns.

00:13:44.140 --> 00:13:46.820
And of course, I'll put that in the show notes as well.

00:13:46.980 --> 00:13:47.740
Yes, that would be great.

00:13:47.740 --> 00:13:53.660
I mean, we also have like a GitHub repository and the whole book is open source released under a Creative Commons license.

00:13:53.660 --> 00:13:56.740
So we're always happy to see contributions.

00:13:56.740 --> 00:14:03.000
And like, yeah, so if people are interested, they would be very welcome to like contribute.

00:14:03.000 --> 00:14:04.520
Yeah, that's excellent.

00:14:04.520 --> 00:14:05.940
Let me pull this up here.

00:14:06.220 --> 00:14:11.340
So in your book, you've broken it into quick math, six sections.

00:14:11.340 --> 00:14:15.420
You have, you sort of categorize the problems that people run into.

00:14:15.420 --> 00:14:16.940
One of them is correctness.

00:14:16.940 --> 00:14:18.520
One is maintainability.

00:14:18.520 --> 00:14:21.900
Maintainability is one that catches up with you later.

00:14:21.900 --> 00:14:27.700
Readability, security, performance, and then you have a special section on Django.

00:14:27.700 --> 00:14:28.860
Yes.

00:14:28.860 --> 00:14:39.660
So yeah, I mean, as I said before, we tried to like just categorize the issues or anti-patterns so that people could like go through individual sections.

00:14:39.660 --> 00:14:50.440
And for example, see what type of problems really affect the correctness of their program and what type of anti-patterns only affect, for example, the readability or the performance.

00:14:50.440 --> 00:14:53.380
So let's maybe look at a couple of them.

00:14:53.380 --> 00:14:57.720
I'll just sort of scan through here and just tell people the kind of stuff that's in there.

00:14:57.720 --> 00:14:58.180
Sure.

00:14:58.800 --> 00:15:08.400
One that I think is somewhat common when you're new to programming in Python is this one you call bad accept clauses order.

00:15:08.400 --> 00:15:09.940
Can you talk about that?

00:15:09.940 --> 00:15:10.700
Yes.

00:15:10.700 --> 00:15:20.300
A bad accept clauses order basically means that you put an accept statement, that you have a sequence of multiple accept statements,

00:15:20.300 --> 00:15:27.520
where the first one of those statements is already more generic or general than the second one.

00:15:27.840 --> 00:15:31.040
So basically your second accept clause will never get executed.

00:15:31.040 --> 00:15:33.720
And that's kind of really easy to get wrong.

00:15:33.720 --> 00:15:36.880
It also happens to me a lot of the time.

00:15:36.880 --> 00:15:37.400
Yeah.

00:15:37.400 --> 00:15:37.440
Yeah.

00:15:37.440 --> 00:15:41.680
It's not obvious, especially if you're going back and adding these additional accept clauses.

00:15:41.680 --> 00:15:51.860
So maybe you say try, you're going to do some code and then you say accept exception as X and then accept type error as TE.

00:15:51.860 --> 00:15:52.540
Right?

00:15:52.540 --> 00:15:56.900
That type error will never get run because the exception is the base class of type error.

00:15:56.900 --> 00:16:02.880
Even worse than this, a pattern is probably an empty accept plug without also an exception type.

00:16:02.880 --> 00:16:06.200
I think that's kind of the most evil Python anti-pattern that exists.

00:16:06.200 --> 00:16:23.520
This episode is brought to you by CodeShip.

00:16:24.180 --> 00:16:32.380
CodeShip has launched organizations, create teams, set permissions for specific team members, and improve collaboration in your continuous delivery workflow.

00:16:32.380 --> 00:16:38.260
Maintain centralized control over your organization's projects and teams with CodeShip's new organizations plan.

00:16:38.260 --> 00:16:43.800
And as Talk Python listeners, you can save 20% off any premium plan for the next three months.

00:16:44.060 --> 00:16:47.320
Just use the code TALKPython, all caps, no spaces.

00:16:47.320 --> 00:16:53.040
Check them out at CodeShip.com and tell them thanks for supporting the show on Twitter where they're at CodeShip.

00:16:53.040 --> 00:17:07.240
I have to admit, sometimes I use it, but it's really bad because it swallows up all your exceptions and it doesn't give you any useful information about what happens in your code.

00:17:08.160 --> 00:17:15.020
Yeah, maybe if you're lucky, the empty accept clause at least has like a logging statement or something.

00:17:15.020 --> 00:17:15.740
Yeah, that's already good, yeah.

00:17:15.740 --> 00:17:20.880
Yeah, but if it has accept pass, that's pretty bad.

00:17:20.880 --> 00:17:21.900
That's really easy, yeah.

00:17:21.900 --> 00:17:24.920
I spend many hours debugging stuff like this, yeah.

00:17:24.920 --> 00:17:27.400
Most of the time I wrote that myself.

00:17:27.400 --> 00:17:32.520
Yeah, I like to think of that one as the intern clause.

00:17:32.520 --> 00:17:33.240
Yes.

00:17:33.240 --> 00:17:35.420
Because they're like, oh, this isn't working.

00:17:35.520 --> 00:17:38.540
Oh, I can just put this in here and my code is now reliable.

00:17:38.540 --> 00:17:40.140
I'll be out of here in three months.

00:17:40.140 --> 00:17:42.140
Let's see.

00:17:42.140 --> 00:17:44.100
There's some other interesting ones.

00:17:44.100 --> 00:17:49.220
Under future import is not the first non-doc string statement.

00:17:49.220 --> 00:17:50.820
What's the problem with that?

00:17:50.820 --> 00:18:02.760
Yeah, so this is just a convention of the Python interpreter because you can use these so-called future imports to import functionality that is usually not available in your Python version.

00:18:03.040 --> 00:18:05.900
For example, one thing that is often used are absolute imports.

00:18:05.900 --> 00:18:11.960
And the thing with the future imports is that they have to be the first statement in your file.

00:18:11.960 --> 00:18:19.420
So you cannot have auto code and then afterwards a future import because the Python interpreter needs to load these ones right at the beginning.

00:18:19.420 --> 00:18:26.420
So, I mean, this is kind of an easy pattern to catch, but still, sometimes it happens that you have this in your code.

00:18:26.880 --> 00:18:31.660
I mean, it's also something that you would notice really easily because your program wouldn't run even.

00:18:31.660 --> 00:18:32.700
Yeah, for sure.

00:18:32.700 --> 00:18:43.860
Another one that you can tell that people are coming from somewhere else and they haven't quite fully gotten the Pythonic idiomatic style is implementing Java style or C++ style getters and setters.

00:18:44.000 --> 00:18:44.240
Oh, yeah.

00:18:44.240 --> 00:18:45.320
Yeah, I like this one a lot.

00:18:45.320 --> 00:18:47.740
We also see it often in our analysis.

00:18:48.160 --> 00:18:56.040
And so this is basically when you write set something and get something function in a class, which you would do in Java.

00:18:56.040 --> 00:19:02.740
Whereas in Python, the appropriate or the correct way to do this would be to use so-called class properties.

00:19:02.740 --> 00:19:13.040
We can just have just access an attribute of a class using a function, which looks like an attribute access, so to say.

00:19:13.040 --> 00:19:13.820
Right.

00:19:13.820 --> 00:19:15.360
Just use the app property decorator.

00:19:15.360 --> 00:19:15.540
Yeah.

00:19:16.100 --> 00:19:16.820
Yes, exactly.

00:19:16.820 --> 00:19:17.260
Yeah.

00:19:17.260 --> 00:19:18.660
Yeah.

00:19:18.660 --> 00:19:21.320
Then there's others that are less obvious to me.

00:19:21.320 --> 00:19:25.560
One of them is if I'm using, I'm not using the default dict.

00:19:25.560 --> 00:19:27.080
What's the story of that one?

00:19:27.080 --> 00:19:29.800
Oh, so this is more a recommendation than a real antipodent.

00:19:30.340 --> 00:19:39.020
Python has this great collections library, which contains many data structures that can help you to make your code easier to understand.

00:19:39.020 --> 00:19:41.060
And the default dict is one of them.

00:19:41.060 --> 00:19:46.200
And it's one of my favorite ones and the one I probably use most in my code.

00:19:46.860 --> 00:19:56.120
And what it is, is a dictionary that basically initializes the value of a given key to something that you can pass in as a function.

00:19:56.120 --> 00:20:03.440
So the typical use case for this would be that you have a dictionary with a list inside.

00:20:03.800 --> 00:20:14.140
And so if you want to create an entry for some key that you haven't seen before, normally what you would need to do is to check if the key has been set in the dictionary before.

00:20:14.140 --> 00:20:18.000
And if not, create a list, an empty list in that place.

00:20:18.400 --> 00:20:20.180
And then append the element to the list.

00:20:20.180 --> 00:20:22.340
Yeah, so it's like three or four lines of code, right?

00:20:22.340 --> 00:20:25.660
But the default dictionary basically does, yes, so that's four lines of code.

00:20:25.660 --> 00:20:30.520
And the default dictionary basically does that for you because you can just say default dict of list.

00:20:30.520 --> 00:20:40.420
And then whenever you would like access a given value for a key in the dictionary, it will have created an empty list if there has been no value in it already.

00:20:40.420 --> 00:20:41.780
So really, really useful.

00:20:42.840 --> 00:20:50.100
And I did not know this when I started writing Python, but it really makes your code much more readable.

00:20:50.100 --> 00:20:50.700
Right.

00:20:50.700 --> 00:20:52.780
Does that come from the collections module?

00:20:52.780 --> 00:20:54.300
Default dict?

00:20:54.300 --> 00:20:55.240
Yes, exactly.

00:20:55.240 --> 00:20:55.680
Yeah.

00:20:55.680 --> 00:20:56.140
Yeah.

00:20:56.140 --> 00:20:56.440
Yeah.

00:20:56.440 --> 00:20:59.100
So it's not immediately imported, but it's easy to get to.

00:20:59.100 --> 00:21:00.360
Yes.

00:21:00.360 --> 00:21:01.700
It's in the standard library.

00:21:01.700 --> 00:21:02.020
Yeah.

00:21:02.020 --> 00:21:06.180
So another one that you have, let's move on to the next section of maintainability.

00:21:06.760 --> 00:21:13.780
And one of them I think is probably pretty common is you should not be using wildcard imports.

00:21:13.780 --> 00:21:16.440
So from math import star.

00:21:16.440 --> 00:21:19.040
It's so much easier to type.

00:21:19.040 --> 00:21:20.620
Why shouldn't I just do that all the time?

00:21:20.620 --> 00:21:21.220
Yeah.

00:21:21.220 --> 00:21:27.480
It's also a mistake which I made like a lot in the beginning when I started writing Python code.

00:21:27.480 --> 00:21:34.240
And it's pretty insidious because normally it doesn't break anything.

00:21:34.240 --> 00:21:39.480
But in some circumstances, you can overwrite functions that you have imported before.

00:21:39.480 --> 00:21:45.320
And so the behavior of your program changes in like really unexpected ways, I would say.

00:21:45.960 --> 00:21:50.480
And so today I always try to use only qualified imports.

00:21:50.480 --> 00:21:58.700
So either import the module and then use it as a name when I access the variable or explicitly name the things that I import from a module.

00:21:58.700 --> 00:21:59.180
Yeah.

00:21:59.180 --> 00:22:00.140
I'm with you on that.

00:22:00.140 --> 00:22:05.540
I try to, if it's not too much writing, try to use the name so it's really clear where the type is coming from.

00:22:05.540 --> 00:22:07.740
But sometimes, you know, it's like, well.

00:22:07.740 --> 00:22:08.860
Yeah.

00:22:08.860 --> 00:22:09.180
I know.

00:22:09.180 --> 00:22:10.160
I do it sometimes too.

00:22:10.160 --> 00:22:12.800
But, you know, more as like an explicit import.

00:22:12.800 --> 00:22:13.080
Yeah.

00:22:13.560 --> 00:22:17.940
So another area that you guys did a lot of work on was readability.

00:22:17.940 --> 00:22:26.160
And I think one of the ones that's, I don't know how easy this is for your system to check automatically.

00:22:26.160 --> 00:22:34.260
But certainly if you come from a language like Java or C# or C++ is not in your way of thinking.

00:22:34.260 --> 00:22:34.920
Right.

00:22:34.920 --> 00:22:38.140
Even though it's one of the core tenets of Python.

00:22:38.140 --> 00:22:40.880
And that's ask for permission instead of for forgiveness.

00:22:42.100 --> 00:22:45.740
Asking for permission is the anti-pattern rather than for forgiveness.

00:22:45.740 --> 00:22:46.260
Yes, exactly.

00:22:46.260 --> 00:22:46.640
Yeah.

00:22:46.640 --> 00:22:47.700
Can you tell people about that?

00:22:47.700 --> 00:22:59.740
So usually when you write Python code, the preferred way is to like assume that everything will go as you expect and not perform any checks.

00:22:59.740 --> 00:23:05.620
But rather catch an exception if some part of your program chose one.

00:23:05.620 --> 00:23:07.420
And like handle that exception.

00:23:07.420 --> 00:23:12.920
And then react to like behavior which you didn't anticipate, so to say.

00:23:12.920 --> 00:23:16.520
A good example is always like when you're opening a file.

00:23:16.860 --> 00:23:23.000
You could think of like checking if the path exists and then open the file at the path.

00:23:23.000 --> 00:23:29.980
But I mean between the two calls, like check of existence of the file and the opening, that could actually happen and stuff.

00:23:29.980 --> 00:23:35.180
And so your program could still crash and you will have an exception handler for that then.

00:23:35.560 --> 00:23:39.980
So the preferred way here is to just assume that the file is there and try to open it.

00:23:39.980 --> 00:23:46.260
And if you get an error because the file does not exist or something else is wrong, then handle that exception, so to say.

00:23:46.260 --> 00:23:46.620
Yeah.

00:23:46.740 --> 00:23:48.540
I think that's pretty key.

00:23:48.540 --> 00:23:53.880
Another example is just, you know, you're trying to think of all the use cases of how something could go bad.

00:23:53.880 --> 00:23:57.460
And you end up with like five or six if tests.

00:23:57.460 --> 00:24:00.960
And then finally you're going to actually perform the thing.

00:24:00.960 --> 00:24:02.300
And you may have forgotten one of those.

00:24:02.300 --> 00:24:03.160
Or there may be something else.

00:24:03.160 --> 00:24:04.720
So imagine I'm going to call a web service.

00:24:04.720 --> 00:24:05.360
Yeah, exactly.

00:24:05.360 --> 00:24:07.000
I've got to make sure the network is on.

00:24:07.000 --> 00:24:11.060
I've got to make sure that the address is well formed.

00:24:11.520 --> 00:24:15.000
And then, you know, I've got to make sure my data, I'm going to send it as well for them.

00:24:15.000 --> 00:24:17.780
And then it still may be the case that the DNS name doesn't look up.

00:24:17.780 --> 00:24:19.980
I mean, there's so many things that could go wrong.

00:24:19.980 --> 00:24:24.420
Just try it and somewhere higher up where it makes sense, catch it and handle it, right?

00:24:24.420 --> 00:24:24.700
Yeah.

00:24:24.700 --> 00:24:26.440
That's what you're recommending there.

00:24:26.440 --> 00:24:39.040
Another thing which I did quite often in the beginning when I learned Python was to use, for example, the non variable to like indicate like a special that something happened inside a function, for example,

00:24:39.040 --> 00:24:42.260
that something didn't exist and function couldn't do what it was supposed to do.

00:24:42.260 --> 00:24:47.940
And then basically check for this non value in the code that called the function.

00:24:47.940 --> 00:25:02.040
And this, of course, makes it also very complicated to like handle these kind of errors because now the function does not return only one type of value, but two different ones, like non, for example, and like a numerical value or something.

00:25:02.440 --> 00:25:18.460
And that's also a very good case where you could like ask for forgiveness instead of permission by just like having a function that throws an exception when something goes wrong and then catch that exception in the calling block of the function instead of like looking for a magic non value or something.

00:25:19.140 --> 00:25:32.920
And the thing about that is that complexity that you're adding by having a return of none or a real value or some sentinel value that is indicating the wrong value means you propagate that complexity up through the callers and then it propagates up higher.

00:25:32.920 --> 00:25:33.260
Exactly.

00:25:33.420 --> 00:25:38.140
It's like you forced this style onto everybody up the chain.

00:25:38.140 --> 00:25:38.660
That's bad.

00:25:38.660 --> 00:25:50.300
So another one that you have that I think is pretty common for people who are new to Python, but quickly they get over it, is using an un-Pythonic loop or a non-Pythonic loop.

00:25:50.300 --> 00:25:53.000
What's a non-Pythonic loop for the listeners?

00:25:53.000 --> 00:26:05.620
So, I mean, many people, me included, that come, for example, from like a C background, they're used to writing loops where they initialize some loop variable like X.

00:26:05.620 --> 00:26:06.860
They set them to zero.

00:26:06.860 --> 00:26:13.460
Then they increase the value of that variable to, for example, the length of an array.

00:26:13.640 --> 00:26:23.780
And inside the loop, they then fetch the value of the array element using the index operator, so to say.

00:26:23.780 --> 00:26:36.580
And in Python, of course, you can just like iterate directly over a list, for example, and not use a range to like first get the indices of the list and then only retrieve the value of the inside the loop.

00:26:36.580 --> 00:26:47.720
I mean, this is only a small thing, but it also makes your code much more readable and it like helps you to avoid errors because it involves less variables and less things that can go wrong, so to say.

00:26:47.720 --> 00:26:49.040
Yeah, that makes a lot of sense.

00:26:49.040 --> 00:26:53.640
So the next section, security, is actually pretty simple.

00:26:53.640 --> 00:27:01.940
Yes, I have to say we have to work on a security section because right now we only have, I think, one entry, which is the use of exec.

00:27:02.620 --> 00:27:10.580
And in our analysis tool, we flagged that by default because it's not always an anti-pattern or something that you should avoid.

00:27:10.580 --> 00:27:21.060
But it's, of course, a huge, huge opening in your system and it can like be used easily to like execute like untrusted code.

00:27:21.260 --> 00:27:33.740
So I think it's not generally an anti-pattern, but if you use it, you should definitely be really careful with it and like think a lot about which code or which things you're passing into exec.

00:27:33.740 --> 00:27:35.100
Yeah, absolutely.

00:27:35.100 --> 00:27:39.880
Because unfortunately, Python does not have a real sandbox as well, a safe sandbox.

00:27:39.880 --> 00:27:49.980
So, I mean, there are some things that you can do to make it a bit more safe to use exec, but there's nothing that can prove like 100% security in that case.

00:27:49.980 --> 00:27:55.340
Yeah, to a large degree, it comes down to how much do you trust that string input that you're passing there, right?

00:27:55.340 --> 00:27:56.300
Yes, exactly.

00:27:56.300 --> 00:27:56.940
Yeah.

00:27:56.940 --> 00:28:02.080
And if it's entered into a community forum, it probably shouldn't be exec.

00:28:02.080 --> 00:28:03.400
Yes.

00:28:03.920 --> 00:28:09.060
It would be basically the Python equivalent of a SQL injection attack.

00:28:09.060 --> 00:28:10.220
Yes.

00:28:10.220 --> 00:28:11.500
Something to that effect.

00:28:11.500 --> 00:28:12.640
Okay.

00:28:12.640 --> 00:28:16.140
Some other ones that were interesting in your performance area.

00:28:16.140 --> 00:28:21.300
You have one using key and list to check if a key is contained in a list.

00:28:21.300 --> 00:28:22.240
What's the story of that one?

00:28:22.240 --> 00:28:23.920
Yeah.

00:28:23.920 --> 00:28:33.900
I had to say the performance section is also pretty small right now, but this pattern has to do with the complexity or the computational complexity of an operation.

00:28:33.900 --> 00:28:51.540
Because if you, for example, try to find out if a certain value is in a long list, in our example, we have, for example, a list of several numbers, and you want to find out if, for example, a tree is in that list, then the code that checks this will have a linear runtime.

00:28:51.540 --> 00:28:55.880
So it will take longer, the longer the list is, so to say.

00:28:55.880 --> 00:29:08.680
And if you would use a dict instead, where you have just a mapping between the values that you have in a list and a true value, for example, then you could perform the same check in constant time.

00:29:08.680 --> 00:29:09.680
So order of one.

00:29:11.100 --> 00:29:20.900
I mean, it's also like a small thing, but if you have really long, large data structures, which, for example, I had in my PhD when analyzing data, this can make a huge difference in the wrong time.

00:29:20.900 --> 00:29:21.340
You're right.

00:29:21.340 --> 00:29:23.100
It can make a tremendous difference.

00:29:23.100 --> 00:29:29.520
I'll tell you a real world story where this completely changed my perspective on this.

00:29:39.100 --> 00:29:42.200
This episode is brought to you by Hired.

00:29:42.200 --> 00:29:48.660
Hired is a two-sided, curated marketplace that connects the world's knowledge workers to the best opportunities.

00:29:48.660 --> 00:29:57.820
Each offer you receive has salary and equity presented right up front, and you can view the offers to accept or reject them before you even talk to the company.

00:29:57.820 --> 00:30:04.180
Typically, candidates receive five or more offers in just the first week, and there are no obligations ever.

00:30:04.180 --> 00:30:06.280
Sounds pretty awesome, doesn't it?

00:30:06.280 --> 00:30:08.340
Well, did I mention there's a signing bonus?

00:30:08.920 --> 00:30:16.760
Everyone who accepts a job from Hired gets a $2,000 signing bonus, and as Talk Python listeners, it gets way sweeter.

00:30:16.760 --> 00:30:24.320
Use the link Hired.com slash Talk Python To Me, and Hired will double the signing bonus to $4,000.

00:30:24.320 --> 00:30:26.040
Opportunity's knocking.

00:30:26.040 --> 00:30:29.660
Visit Hired.com slash Talk Python To Me and answer the call.

00:30:29.660 --> 00:30:43.160
I was working on this project involving eye tracking.

00:30:43.160 --> 00:30:45.560
Like, not the letter I, but, you know, where you're looking.

00:30:46.400 --> 00:30:52.100
And this eye tracking system was collecting data at 250 hertz, so 250 times a second.

00:30:52.100 --> 00:30:58.760
And we were trying to do real-time analysis on this, which meant we had four milliseconds between the times that sample came in.

00:30:58.760 --> 00:31:01.460
It had to, like, really be quick and efficient.

00:31:02.120 --> 00:31:06.080
And we were doing some stuff that we had ported over from MATLAB.

00:31:06.080 --> 00:31:08.800
And we were doing basically what you said.

00:31:08.800 --> 00:31:11.860
We had kind of this, like, running buffer of data.

00:31:11.860 --> 00:31:14.500
And we would have to seek back into it and find certain elements.

00:31:14.500 --> 00:31:17.700
And then we would apply that forward over a certain amount of time.

00:31:18.300 --> 00:31:20.420
And we ran it, and the code was too slow.

00:31:20.420 --> 00:31:22.120
We're like, oh, no, it's too slow.

00:31:22.120 --> 00:31:27.320
If we can't make it go four milliseconds, it's not going to be real-time.

00:31:27.320 --> 00:31:28.400
It's just that simple, right?

00:31:28.400 --> 00:31:32.600
And so there was just tons of complicated math.

00:31:32.600 --> 00:31:36.180
And, like, we really don't want to try to optimize that.

00:31:36.180 --> 00:31:38.840
We're like, it's like wavelet decomposition.

00:31:38.840 --> 00:31:42.240
And so it was, like, on the very edge of even understanding the math.

00:31:42.240 --> 00:31:44.280
And so optimizing it was a bad idea.

00:31:45.120 --> 00:31:49.360
And after running some performance analysis, profiling stuff on the code,

00:31:49.360 --> 00:31:53.760
it turned out 80% of the time was spent looking for an item in the list.

00:31:53.760 --> 00:31:54.520
Oh, I see.

00:31:54.520 --> 00:31:59.680
So we got it to go, like, five times faster by switching to a dictionary.

00:31:59.680 --> 00:32:01.840
And it was, like, basic.

00:32:01.840 --> 00:32:02.680
That's great.

00:32:02.680 --> 00:32:03.480
That's a quick win, yeah.

00:32:03.480 --> 00:32:04.360
It was so little work.

00:32:04.360 --> 00:32:08.360
And we got to avoid optimizing wavelet decomposition math, which is fantastic.

00:32:08.360 --> 00:32:10.260
That's awesome, yeah.

00:32:10.260 --> 00:32:12.320
Yeah, so it sounds really simple.

00:32:12.320 --> 00:32:14.080
Yeah, no, that's a perfect example.

00:32:14.260 --> 00:32:16.520
I mean, yeah, small things can make a big difference, yeah.

00:32:16.520 --> 00:32:17.580
Yeah, they sure can.

00:32:17.580 --> 00:32:18.260
That's cool.

00:32:18.260 --> 00:32:20.140
The last section is Django.

00:32:20.140 --> 00:32:22.060
And I'm more of a micro-framework guy.

00:32:22.060 --> 00:32:23.020
I haven't done a lot of Django.

00:32:23.020 --> 00:32:24.860
Probably have a show on that coming up at some point.

00:32:24.860 --> 00:32:27.100
But maybe you could talk about some of the highlights there for us.

00:32:27.100 --> 00:32:31.920
Yeah, so we added a Django section so that a lot of people were using a project.

00:32:31.920 --> 00:32:38.200
I mean, a lot of the Python projects out there are web projects using Django or Flask or something else.

00:32:38.200 --> 00:32:48.900
And so we wanted to help people also not only writing better Python code in general, but also getting more proficient at libraries like Django or Flask or anything else.

00:32:49.560 --> 00:32:54.380
And so the Django section is basically organized in the same way as the main section.

00:32:54.380 --> 00:32:57.340
So we have maintainability, security, correctness, etc.

00:32:57.940 --> 00:33:05.060
And the only difference is that we are only talking about stuff that is really specific to the Django framework.

00:33:05.060 --> 00:33:17.640
So we're telling you how to import modules, how to use certain fields and models, which variables you should have in your config and which variables should not have in your config under some conditions.

00:33:17.700 --> 00:33:29.840
And also something that is really helpful to many people is we show you which things you would need to change actually in your code if, for example, you wanted to migrate from Django 1.6 to Django 1.8.

00:33:29.840 --> 00:33:31.460
Right. Oh, that's really helpful.

00:33:31.460 --> 00:33:33.680
There's a lot of stuff in there that's not obvious.

00:33:33.680 --> 00:33:38.480
And I think a lot of people learn Django by going to the tutorial and going through it and go, now I know Django.

00:33:38.480 --> 00:33:39.300
Yeah, exactly.

00:33:39.300 --> 00:33:41.520
I mean, Django is changing really fast.

00:33:41.520 --> 00:33:48.300
I mean, the community is really awesome and they're putting out a lot of new features and new versions.

00:33:48.300 --> 00:33:58.820
And I think it's good to have like this resource where you can see what has changed and how you would need to change your code in order to make it work with the latest version.

00:33:58.820 --> 00:34:06.480
And I mean, normally you want to be on the latest version because it's more secure and more features and probably better than the precedent one.

00:34:06.480 --> 00:34:08.080
Yeah, definitely.

00:34:08.080 --> 00:34:11.520
When you're running a website, you definitely want to stay on the latest version of your framework.

00:34:11.520 --> 00:34:12.280
Exactly.

00:34:12.280 --> 00:34:13.300
Okay.

00:34:13.300 --> 00:34:23.020
So this project, the book you said is on GitHub and your general GitHub is just, it's github.com/quantified code, all lowercase.

00:34:23.020 --> 00:34:23.240
Yeah.

00:34:23.240 --> 00:34:24.440
Exactly.

00:34:24.440 --> 00:34:25.140
Yeah.

00:34:25.140 --> 00:34:37.060
So something I found up there when I went to check out your anti-patterns project on GitHub was you also have this thing called a repository called xMachina.

00:34:37.400 --> 00:34:38.040
Ah, yeah.

00:34:38.040 --> 00:34:38.380
Okay.

00:34:38.380 --> 00:34:39.440
That's pretty cool.

00:34:39.440 --> 00:34:41.520
Can you tell just people really briefly what that is?

00:34:41.520 --> 00:34:42.160
It looks awesome.

00:34:42.160 --> 00:34:51.740
I have to say this is something that my co-founder, Christoph, did because in the movie xMachina, there's a snippet of Python code that you can see on the screen.

00:34:52.280 --> 00:34:57.020
And he just wrote down the code because he was interested in what it would do.

00:34:57.020 --> 00:35:02.380
And it actually prints out an ISBN number of a book that is mentioned in the movie.

00:35:03.320 --> 00:35:09.820
And he also ran it through our static analysis software and he found that actually it's not like PEP 8 compliant and there's some other things in it.

00:35:09.820 --> 00:35:13.620
So it probably shouldn't be security code, which it was in the movie.

00:35:13.620 --> 00:35:14.380
Sure.

00:35:14.720 --> 00:35:20.580
Well, you know, the fact that you can take code from a movie and it even runs is already a good deal.

00:35:20.580 --> 00:35:21.200
Yeah, it's great.

00:35:21.200 --> 00:35:23.260
I mean, it's really good to see.

00:35:23.260 --> 00:35:26.040
Awesome to see that Python is used in movies now.

00:35:26.040 --> 00:35:26.820
Yeah, yeah.

00:35:26.820 --> 00:35:28.320
XMachina looks really cool.

00:35:28.440 --> 00:35:29.440
I have not seen it yet.

00:35:29.440 --> 00:35:30.760
Yeah, great movie.

00:35:30.760 --> 00:35:33.760
I would like to talk about it, but I don't want to say any spoilers.

00:35:33.760 --> 00:35:34.760
Oh, yeah, yeah.

00:35:34.760 --> 00:35:38.460
I'll put a link to the trailer on YouTube in the show notes for everyone.

00:35:38.460 --> 00:35:40.140
They can check it out and go see it for themselves.

00:35:40.140 --> 00:35:41.240
Awesome.

00:35:41.240 --> 00:35:45.920
So another thing that you guys have up there is something called Code is Beautiful.

00:35:45.920 --> 00:35:48.500
What's the story with that?

00:35:48.500 --> 00:35:49.500
Exactly.

00:35:49.500 --> 00:35:58.720
So in Quantified Code, we don't only do code analysis, but we're also thinking about new or better ways to visualize code.

00:35:58.720 --> 00:36:11.440
Because for us, dealing with a large code base always involves getting an overview of the whole thing and seeing where, for example, problems are or where work would need to be done.

00:36:11.440 --> 00:36:14.320
And so visualizing the code is a great way to do that.

00:36:14.800 --> 00:36:24.680
And we always liked things like, for example, Cityscape visualizations, but we didn't find anything that was open source or available to the public, so to say.

00:36:24.680 --> 00:36:33.340
And that's why we said, hey, let's develop some cool visualizations of Python code and give them for free to the community.

00:36:33.340 --> 00:36:36.000
So we started this code, this open source project.

00:36:36.000 --> 00:36:39.700
And we have currently three different visualizations of source code.

00:36:39.880 --> 00:36:50.780
One is like a Cityscape visualization, which is like a 3D version of your code where you can, so to say, zoom in and see at a glance where, for example, the most complex part of your code are.

00:36:50.780 --> 00:36:51.720
That's really cool.

00:36:51.720 --> 00:36:54.040
And you should point out to people that it's live.

00:36:54.040 --> 00:37:00.540
You can rotate it and hover over these little buildings or skyscrapers, and it'll tell you what segment of your code that is.

00:37:00.540 --> 00:37:02.860
What do the colors mean in your visualizations?

00:37:03.500 --> 00:37:08.580
So in the visualization, currently, the color encodes the complexity on the code.

00:37:08.580 --> 00:37:13.560
So it's basically the cyclomatic complexity divided by the number of lines.

00:37:13.560 --> 00:37:14.840
That is fantastic.

00:37:14.840 --> 00:37:21.520
So the cyclomatic complexity is just the number of branches in the code, so to say.

00:37:21.520 --> 00:37:27.160
Roughly, you can think of it like the number of unit tests you would need to test a given piece of code.

00:37:27.160 --> 00:37:27.460
Right.

00:37:27.520 --> 00:37:31.240
How many decision points are there in a given function in a class or something like that.

00:37:31.240 --> 00:37:37.180
So I can create this cool cityscape, and the red buildings are more bad.

00:37:37.840 --> 00:37:47.160
And if I'm going to look at somebody's code, I guess maybe looking at the height and width of a building also tells me how much is monolithic.

00:37:47.160 --> 00:37:47.460
Yeah, exactly.

00:37:47.460 --> 00:37:57.060
So the area is the number of lines in a given file, and the height is just the total complexity of the file.

00:37:57.060 --> 00:37:58.080
Yeah, okay.

00:37:58.080 --> 00:37:59.100
Very cool.

00:37:59.100 --> 00:38:03.800
And then you also have two other visualizations that are, I would say, more standard.

00:38:04.460 --> 00:38:08.340
Yes, so they're like 2D, a bit more down-to-earth visualizations.

00:38:08.340 --> 00:38:18.540
And, yeah, we have them because many browsers still don't support WebGL, and we wanted to have something that, like, everyone can use, so to say.

00:38:18.540 --> 00:38:19.340
Right.

00:38:19.340 --> 00:38:21.900
And they're all interactive with D3, right?

00:38:21.900 --> 00:38:23.720
So you can sort of explore them and so on.

00:38:23.720 --> 00:38:25.000
Yeah, exactly.

00:38:25.000 --> 00:38:30.340
And they're also open source, so you can download them in GitHub, use them in your own projects, and modify them.

00:38:30.340 --> 00:38:31.080
Excellent.

00:38:31.080 --> 00:38:33.820
Yeah, you have a live demo that people can go check out, right?

00:38:33.920 --> 00:38:34.800
And what are you analyzing?

00:38:34.800 --> 00:38:37.020
Are you analyzing Django there, or what are you analyzing?

00:38:37.020 --> 00:38:45.200
Oh, we have a lot of projects on our platform, about, like, 10,000 public GitHub projects that we constantly analyze.

00:38:45.200 --> 00:38:50.840
And you can basically go there and see if your project has any problems or if you could improve anything.

00:38:50.840 --> 00:38:54.880
And you can also get visualizations of all the popular Python projects there.

00:38:54.880 --> 00:38:56.240
Oh, that's awesome.

00:38:56.240 --> 00:39:00.700
Is there a way to hook that visualization into a continuous integration story?

00:39:00.700 --> 00:39:05.960
So, like, every time I push to a branch, a build is kicked off, and then an analysis is kicked off and saved.

00:39:05.960 --> 00:39:07.040
Have you guys done this?

00:39:07.040 --> 00:39:07.780
Oh, yeah, you can do that.

00:39:07.780 --> 00:39:07.940
Really?

00:39:07.940 --> 00:39:11.760
You can just sign up for free on our website, and you can add your GitHub project.

00:39:11.760 --> 00:39:16.680
And every time you will make a commit, we will analyze your project, and then you can get the visualizations on our website.

00:39:17.680 --> 00:39:19.860
And it's, of course, free for open-source projects.

00:39:19.860 --> 00:39:21.100
Yeah, yeah, that's really cool.

00:39:21.100 --> 00:39:22.620
So, that's quantifiedcode.com.

00:39:22.620 --> 00:39:23.560
Yeah, very nice.

00:39:23.560 --> 00:39:27.280
Yeah, I really like what you guys are doing there with that one.

00:39:27.280 --> 00:39:28.040
Thanks.

00:39:28.040 --> 00:39:28.400
Yeah.

00:39:28.400 --> 00:39:28.680
Yeah.

00:39:29.560 --> 00:39:32.820
So, I have a few more questions before we call it a show.

00:39:32.820 --> 00:39:37.260
One is, what are the worst anti-patterns that you see?

00:39:37.260 --> 00:39:42.540
Like, we went through a couple of them, but are there some that you go, oh, my gosh, if you do this, then it's really bad?

00:39:42.540 --> 00:39:50.660
So, I mean, as I said before, the empty try-accept statement is one of the worst things to debug, I think.

00:39:50.660 --> 00:39:58.140
And apart from that, I think it's not the simple anti-patterns that are, like, killing software projects.

00:39:58.140 --> 00:40:11.380
It's mostly, like, complexity and, like, changing code too fast and without using, having a solid idea of what you're doing and without, like, writing unit tests.

00:40:11.380 --> 00:40:19.640
So, I think there's no really easy recipe to say, okay, if you avoid this and this and this, then you will write very good code.

00:40:19.640 --> 00:40:34.720
But I think it's, like, mostly about, like, controlling your complexity and, like, constantly thinking about your code structure and thinking how you can improve it and also discussing it with other people and doing manual code reviews and automated code reviews.

00:40:35.420 --> 00:40:41.340
So, yeah, for me, it's like, as a software development, our daily job is to fight complexity.

00:40:41.340 --> 00:40:43.560
Yeah, I think you're right.

00:40:43.560 --> 00:40:52.460
One of the problems that people end up with is they end up with complex, highly cyclomatic, you know, high cyclomatic complexity and all that given to them.

00:40:52.460 --> 00:40:56.220
And then they're like, well, if I had written this, I would be beautiful.

00:40:56.220 --> 00:40:57.600
But, of course, I didn't.

00:40:57.600 --> 00:40:58.800
And so now what do I do?

00:40:58.800 --> 00:40:59.600
I'm kind of lost.

00:40:59.600 --> 00:41:04.220
So, two books, other than just using your tools to understand it, two books I'd like to recommend to people.

00:41:04.220 --> 00:41:09.260
One is called Refactoring the Patterns by Joshua Kershavisky.

00:41:09.260 --> 00:41:10.660
Probably destroyed his name.

00:41:10.660 --> 00:41:11.540
Sorry about that, Joshua.

00:41:11.540 --> 00:41:16.540
But that's a really cool book of how I take, like, monolithic ugly code and, like, make it better.

00:41:16.540 --> 00:41:18.840
And the other one is a book by…

00:41:18.840 --> 00:41:19.600
I can really recommend it.

00:41:19.600 --> 00:41:19.980
Yeah.

00:41:19.980 --> 00:41:20.780
Do you know the book?

00:41:20.780 --> 00:41:22.360
Yeah, I know.

00:41:22.360 --> 00:41:22.540
Yeah.

00:41:22.540 --> 00:41:23.800
It's a good one.

00:41:23.800 --> 00:41:36.220
The other one is Working Effectively with Legacy Code by Michael Feathers, which shows you how to take huge monolithic code and slowly break it into more maintainable pieces without breaking it, you know, while keeping it running, basically.

00:41:36.220 --> 00:41:39.720
So, those are some good things to check out if you're interested.

00:41:39.720 --> 00:41:41.520
There's also Clean Code.

00:41:41.520 --> 00:41:43.960
I just don't remember the name of the order right now.

00:41:43.960 --> 00:41:44.340
I don't know.

00:41:44.340 --> 00:41:44.900
Do you remember?

00:41:44.900 --> 00:41:47.080
That would be Robert C. Martin.

00:41:47.080 --> 00:41:48.320
Yes.

00:41:48.320 --> 00:41:48.660
Okay.

00:41:48.660 --> 00:41:49.580
Uncle Bob.

00:41:49.580 --> 00:41:50.820
Sometimes he goes that.

00:41:50.820 --> 00:41:51.080
Yeah.

00:41:51.080 --> 00:41:56.800
So, that's also a great resource, not specifically in refactoring, but more generally in how to write good code.

00:41:56.800 --> 00:42:01.060
And it's mostly written for Java, but it contains so much wisdom.

00:42:01.060 --> 00:42:03.340
I think I read it, like, five times already.

00:42:03.340 --> 00:42:05.080
So, every time I've done something new.

00:42:05.080 --> 00:42:05.680
Yeah.

00:42:05.680 --> 00:42:08.000
He did a really good book there.

00:42:08.000 --> 00:42:10.640
And I believe he might have written that with his son, Micah, as well.

00:42:10.640 --> 00:42:11.520
So, there might be two authors.

00:42:11.520 --> 00:42:12.800
But, yeah, it's very cool.

00:42:12.800 --> 00:42:14.740
All right, Andreas.

00:42:14.740 --> 00:42:17.800
This has been a really interesting conversation.

00:42:17.800 --> 00:42:19.200
Before I let you off the book…

00:42:19.200 --> 00:42:19.920
Yeah, thanks for the conversation.

00:42:19.920 --> 00:42:20.700
You bet.

00:42:20.700 --> 00:42:26.020
Before I let you off the hook, there's a couple of thoughts and questions I have for you.

00:42:26.020 --> 00:42:26.580
Sure.

00:42:26.580 --> 00:42:27.880
One, what's your…

00:42:27.880 --> 00:42:32.920
You know, there's a bunch of PyPI packages and libraries out there in the world.

00:42:32.920 --> 00:42:35.100
Do you have any favorites that you'd like to recommend to people?

00:42:36.180 --> 00:42:36.960
Mm-hmm.

00:42:36.960 --> 00:42:40.000
So, right now, I use SQLAlchemy a lot.

00:42:40.000 --> 00:42:43.920
And I think it's really, really a great Python package.

00:42:43.920 --> 00:42:46.460
It's perfectly documented.

00:42:46.460 --> 00:42:51.140
And it allows you to do so much magic with SQL.

00:42:51.140 --> 00:42:53.160
And it has allowed…

00:42:53.160 --> 00:42:54.700
It has saved us so much time.

00:42:54.960 --> 00:42:56.780
So, it's really, really a great project.

00:42:56.780 --> 00:43:02.140
And if you're working with SQL databases, you should have a look at that, definitely.

00:43:02.140 --> 00:43:03.380
I totally agree.

00:43:03.380 --> 00:43:08.840
You know, if I'm working with a relational database, I wouldn't use anything else, basically.

00:43:08.840 --> 00:43:10.500
That's how I feel about it, right?

00:43:11.500 --> 00:43:15.760
We actually had Mike Bayer on show five to talk about SQL as well.

00:43:15.760 --> 00:43:19.980
And so, if people are listening now and they want to go get more info on it, we've got a whole show there.

00:43:19.980 --> 00:43:20.420
That's awesome.

00:43:20.420 --> 00:43:20.760
Yeah.

00:43:20.760 --> 00:43:22.320
And what's your favorite editor?

00:43:22.320 --> 00:43:25.080
I mostly use Sublime Text, I have to say.

00:43:25.080 --> 00:43:28.580
But I'm not even a power user, so I'm pretty old-fashioned.

00:43:28.580 --> 00:43:31.820
And sometimes I also use Vim to edit my code.

00:43:31.820 --> 00:43:35.120
But I really don't use many key combinations or so.

00:43:35.120 --> 00:43:35.840
So, I'm really…

00:43:35.840 --> 00:43:39.020
When it comes to developing, writing code, I'm…

00:43:39.020 --> 00:43:39.120
Yeah.

00:43:39.120 --> 00:43:40.280
Yeah.

00:43:40.280 --> 00:43:40.540
Cool.

00:43:40.540 --> 00:43:41.140
Okay.

00:43:41.240 --> 00:43:41.460
Excellent.

00:43:41.460 --> 00:43:41.700
Yeah.

00:43:41.700 --> 00:43:42.980
Sublime is great.

00:43:42.980 --> 00:43:48.740
So, have you thought about selling an e-book version of your little book of anti-patterns?

00:43:48.740 --> 00:43:53.280
Oh, I mean, right now we really don't see this as a commercial project.

00:43:53.280 --> 00:43:56.320
And we just want to make this available to as many people as possible.

00:43:56.320 --> 00:44:01.280
That's why we licensed it using a Creative Commons license.

00:44:01.280 --> 00:44:05.400
So, basically, you can download it, modify it, and do whatever you want with it.

00:44:05.400 --> 00:44:09.680
So, and right now I think we don't have the time to make an e-book version.

00:44:09.780 --> 00:44:13.500
But if somebody would want to do it, he or she could do it.

00:44:13.500 --> 00:44:14.320
Excellent.

00:44:14.320 --> 00:44:15.300
Yeah.

00:44:15.300 --> 00:44:19.860
Seems like you could write a cool script that would generate a little e-pub or something that you could throw at your kiddo.

00:44:21.160 --> 00:44:29.780
I think the documentation swings also supports e-book and mobi outputs.

00:44:29.780 --> 00:44:31.780
So, I think it would be pretty easy to do.

00:44:31.780 --> 00:44:32.200
Oh, yeah.

00:44:32.200 --> 00:44:32.440
Cool.

00:44:32.440 --> 00:44:35.340
It's another good package, which I could recommend, by the way, if you're writing documentation.

00:44:35.340 --> 00:44:35.940
Yeah.

00:44:35.940 --> 00:44:36.140
Yeah.

00:44:36.140 --> 00:44:38.040
That's what's powering your docs and your book.

00:44:38.040 --> 00:44:38.680
Great.

00:44:38.680 --> 00:44:39.140
Yeah.

00:44:39.140 --> 00:44:40.480
Okay.

00:44:40.480 --> 00:44:47.420
And then why don't you just tell us a little bit about what you guys do at Quantified Code so people can check out whatever you're up to in your day job.

00:44:48.420 --> 00:44:52.580
So, I mean, at Quantified Code, what we're doing now is mostly static code analysis.

00:44:53.240 --> 00:44:59.340
But what our big vision is, so to say, is to change the way that people write software.

00:44:59.340 --> 00:45:07.480
Because we think that today we're still using tools that are very similar to the ones that we used like 30 years ago.

00:45:07.480 --> 00:45:22.180
We're still entering text using editors, and we're thinking of code as text files, and we're not making use of all the progress in software development and computer hardware as well that has been made in the last 30 years.

00:45:22.180 --> 00:45:33.940
So, our mission is kind of like to think about new ways to interact with code, to analyze code, and to transform it, and to also make use of all the available data that is out there.

00:45:34.160 --> 00:45:39.460
Because today, most of us develop software using version control systems like Git, et cetera.

00:45:39.460 --> 00:45:45.500
And those systems, they generate an amazing amount of data about every interaction we have with a code base.

00:45:45.500 --> 00:46:00.560
And we can make use of that and basically use that data to find out how you can become a better programmer, how you can be more effective at your job, how you can be more happy maybe, and how you can write code that is more robust.

00:46:00.560 --> 00:46:02.920
So, this is basically what we're trying to do.

00:46:03.380 --> 00:46:16.080
And the first step on this way is, so to say, to develop a new way to do code analysis and help people to avoid some of the most problematic things in their code.

00:46:16.080 --> 00:46:21.080
That's really a great vision and a mission, and you guys are off to a good start.

00:46:21.080 --> 00:46:21.980
Thanks.

00:46:21.980 --> 00:46:22.680
Cool.

00:46:22.680 --> 00:46:23.500
Of course.

00:46:23.500 --> 00:46:25.180
It will take a while until we get there, I guess.

00:46:25.180 --> 00:46:28.340
But, I mean, yeah, somebody should do it.

00:46:28.500 --> 00:46:32.460
Yeah, if you're going to change the world and no one else is doing it, maybe that's you, right?

00:46:32.460 --> 00:46:33.160
That's awesome.

00:46:33.160 --> 00:46:36.720
Okay, well, Andreas, thank you so much for being on the show.

00:46:36.720 --> 00:46:37.820
It's been great to talk.

00:46:37.820 --> 00:46:38.700
Yeah, thanks for having me.

00:46:38.700 --> 00:46:39.500
It was really great.

00:46:39.500 --> 00:46:40.460
Yeah, you bet.

00:46:40.460 --> 00:46:42.280
It's been great to talk about bad code with you.

00:46:42.280 --> 00:46:43.940
Thanks.

00:46:43.940 --> 00:46:44.660
All right.

00:46:44.660 --> 00:46:45.280
See you later.

00:46:45.280 --> 00:46:46.380
See you.

00:46:46.380 --> 00:46:46.740
Bye-bye.

00:46:48.020 --> 00:46:50.520
This has been another episode of Talk Python To Me.

00:46:50.520 --> 00:46:56.120
Today's guest was Andreas Duvez, and this episode has been sponsored by CodeShip and Hired.

00:46:56.120 --> 00:46:58.200
Thank you guys for supporting the show.

00:46:58.200 --> 00:47:03.240
Check out CodeShip at CodeShip.com and thank them on Twitter via at CodeShip.

00:47:03.240 --> 00:47:05.080
Don't forget the discount for listeners.

00:47:05.080 --> 00:47:05.740
It's easy.

00:47:05.740 --> 00:47:08.000
Talk Python, all caps, no spaces.

00:47:08.000 --> 00:47:11.060
Hired wants to help you find your next big thing.

00:47:11.060 --> 00:47:16.900
Visit Hired.com slash Talk Python To Me to get five or more offers with salary and equity

00:47:16.900 --> 00:47:20.800
presented right up front and a special listener signing bonus of $4,000.

00:47:20.800 --> 00:47:29.060
You can find the links from today's show at Talk Python To Me.com slash episodes slash show slash 18.

00:47:29.060 --> 00:47:30.960
Be sure to subscribe to the show.

00:47:30.960 --> 00:47:33.800
Open your favorite podcatcher and search for Python.

00:47:33.800 --> 00:47:35.200
We should be right at the top.

00:47:35.200 --> 00:47:39.940
You can also find the iTunes and direct RSS feeds in the footer of the website.

00:47:40.640 --> 00:47:46.040
Our theme music is the song Developers, Developers, Developers by Corey Smith, who goes by Smix.

00:47:46.040 --> 00:47:48.320
You can hear the entire song on our website.

00:47:48.320 --> 00:47:50.100
This is your host, Michael Kennedy.

00:47:50.100 --> 00:47:50.900
Thanks for listening.

00:47:50.900 --> 00:47:53.260
Smix, take us out of here.

00:48:10.220 --> 00:48:15.140
Developers, Developers, Developers, Developers, Developers, Developers, Developers, Developers.

00:48:15.140 --> 00:48:17.200
you