We hardly encounter anything that didn’t really matter

From Volumetric Regimes
Revision as of 16:47, 4 April 2021 by Possible (talk | contribs)
Jump to navigation Jump to search

We hardly encounter anything that didn’t really matter (a conversation with Phil Langley)

As an architect and computational designer, Phil Langley develops critical approaches to technology and software for architectural practice and spatial design. Our first conversation started from a shared inquiry into MakeHuman[1], the Open Source software project for modeling 3-dimensional humanoid characters.

In the margins of the yearly Libre Graphics meeting in Toronto, we spoke about the way that materiality gets encoded into software, about parametric versus generative approaches, and the symbiotic relationship between algorithms that run simulations and the structure of that algorithm itself. “I think there is a blindness in understanding that the nature of the algorithm effects the nature of the model … The model that you see on your screen is not the model that is actually analyzed.”[2]

Six years later, we ask him about his work for the London based architecture and engineering firm Bryden Woods where he is now responsible for a team that might handle computational design in quite a different way.

A very small ecosystem

Phil Langley: For the creative technologies team that I set up in my company, we hired twenty people doing computational design and they all come from very similar backgrounds: architectural engineering plus a postgraduate or a master’s degree in ‘computational design’. We all have similar skills and are from a narrow selection of academic institutions. It is a very small ecosystem.

I followed a course around 2007 that is similar to what people do now. There’s some of the technology that moves on for sure, but you’re still learning the same kind of algorithms that were there in the 1950s or sixties or seventies. They were already old when I was doing them. You’re still learning some parametrics, some generative design, generative algorithms, genetic algorithms, neural networks and cellular automatisms, it is absolutely a classic curriculum. Same texts, same books, same references. A real echo chamber.

One of the things I hated when I studied was the lack of diversity of thoughts, of criticality around these topics. And also the fact that there’s only a very narrow cross-section of society involved in creating these kinds of techniques. If you ever mentioned the fact that some of these algorithmic approaches came from military research, the response was: “So what?". It wasn’t even that they said that they already knew. They were just like “Nothing to say about that, how can that possibly be relevant?”

How can you say it actually works?

PL: When building the team, I was very conscious about not stepping straight into the use of generative design technologies, because we certainly haven’t matured enough to start the conversation about how careful you have to be when using those techniques. We are working with quite complex situations and so we can’t have a complex algorithm yet because we have too much to understand about the problem itself.

We started with a much more parametric and procedural design approach, that was much more... I wouldn’t say basic... but lots of people in team got quite frustrated at the beginning because they said, we can use this technique, why don’t we just use this? It’s only this year that we started using any kind of generative design algorithms at all. It was forced on us actually, by some external pressures. Some clients demanded it because it becomes very fashionable and they insisted that we did it. The challenges or the problems or the kind of slippage is how to try and build something that uses those techniques, but to do it consciously. And we are not always successful achieving that, by the way.

The biggest thing we were able to achieve is the transparency of the process because normally everything that you pile up to build one of those systems, gets lost. Because it is always about the performance of it, that is what everybody wants to show. They don’t want to tell you how they built it up bit by bit. People just want to show a neural network doing something really cool, and they don’t really want to tell you how they encoded all of the logic and how they selected the data. There are just thousands of decisions to make all the way through about what you include, what you don’t include, how you privilege things and not privilege other things.

At some point, you carefully smooth all of the elements or you de-noise that process so much… You simplify the rules and you simplify the input context, you simplify everything to make it work, and then how can you say that it actually works? Just because it executes and doesn’t crash, is that really the definition of functionality, what sort of truth does it tell you? What answers does it give you?

You make people try to understand what it does and you make people talk about it, to be explicit about each of those choices they make, all those rules, inputs, logics, geometry or data, what they do to turn that into a system. Every one of those decisions you make defines the n-dimensional space of possibilities. And if you take some very complicated input and you can’t handle it in your process and you simplify so much, you’ve already given a shape to what it could possibly emerge as. So one of the things we ended up doing is spending a lot of time on that and we discuss each micro step. Why are we doing it like this? It wasnt always easy for everyone because they didn’t want to think about documenting all the steps.

Yesterday we had a two hour conversation about mesh interpolation and the start of the conversation was a data flow diagram, and one of the boxes just said something like: “We’re just going to press this button and then it turns into a mesh”. And I said: “Woah, wait a minute!” some people thought “What do you mean, it’s just a feature, it’s just an algorithm. It’s just in the software, we can just do it.” And I said, “No way.” That’s even before you get towards building something that acts on that model. I think that’s what we got out of it actually, by not starting with the most let’s say sophisticated approach, it has allowed us to have more time to reflect on what fueled the process.

Decisions have to be made

Possible Bodies: Do you think that transparency can produce a kind of control? Or that ‘understanding’ is somehow possible?

PL: It depends what you mean by control, I would say.

It is not necessarily that you do this in order to increase the efficacy of the process or to ensure you get better results. You don’t do it in order to understand all of the interactions because you can not do that, not really. You can have a simpler algorithmic process, you can have an idea of how it’s operating, there is some truth in that, in the transparency, but you lose that quite quickly as the complexity grows, it’s more to say that you re-balance the idea that you want to see an outcome you like, and therefore then claim that it works. I want to be able to be explicit about everything that I know all the way long. In the end that’s all you have. By making explicit that you have made all these steps, you make clear that decisions have to be made. That at every point you’re intervening in something, and it will have an effect. Almost every one of these things has an effect to a greater or lesser extent and we hardly encounter anything that didn’t really matter. Not even if it was a bug. If it wasn’t really affecting the system, it’s probably because it was a bug in the process rather than anything else.

I think that transparency is not about gaining control of a process in itself, it’s about being honest with the fact that you’re creating something with a generative adversarial network (GAN) or a neural network, whatever it is. That it doesn’t just come from TensorFlow[3], fully made and put into your hand and you just press play.

Getting lost in nice little problems

PL: The point I was trying to make to everyone on the team was, well, if you simplify the mesh so much in order that it’s smooth and so you can handle it in the next process, what kind of reliance can you have on the output?

I’ll tell you about a project that’s sort of quite boring. We are developing an automated process for cable rooting for signaling systems in tunnels. We basically take a point cloud survey of a tunnel and we’re trying to route this cable between obstacles. The tunnel is very small, there is no space, and obviously there’s already a signaling system there. So there are cables everywhere and you can’t take them out while you install the new ones, you have to find a pathway. Normally this would be done manually. Overnight people would go down in the tunnel and spray paint the wall and then photograph it and then come back to the office and try and draw it. So we’re trying do this digitally and automate it in some way. There’s some engineering rules of the cables, that have to be a certain diameter. You can’t just bend them in any direction... it was a really nice geometric problem. The tunnel is a double curvature, and you have these point-clouds ... there were loads of quite nice little problems and you can get lost in it.

PB: It doesn’t sound like a boring project?

PL: No it’s absolutely not boring, it’s just funny. None of us have worked in rail before. No one has ever worked in these contexts. We just turned up and went: “Why’d you do it like that?”

Once you finally get your mesh of the tunnel, what you’re trying to do is subdivide that mesh into geometry again, another nice problem. A grid subdivision or triangles or hexagons, my personal favorite. And then you’re trying to work out, which one of these grid subdivisions contains already a signal box, a cable or another obstruction basically? What sort of degree of freedom do I have to navigate through this? Taking a very detailed sub-millimeter accuracy point-cloud, that you’re reducing into a subdivision of squares, simplifying it right down. And then you turn it into an evaluation. And then you have a path-finding algorithm that tries to join all the bits together within the engineering rules, within how you bend the cable. And you can imagine that by the time you get to start mesh subdivision, if you process that input to death, it’s going to be absolutely meaningless. It will work in the sense that it will run and it will look quite cool, but what do I do with it?

I try to talk about language with everybody

PL: I try to talk a little bit about language with everybody. I’m trying not to overburden everybody with all of my predilections. I can’t really impose anything on them. Language is a big thing, like explaining Genetic Algorithms with phrases like , “So this is the population that would kill everybody, that’s like unsuitable or invalid” And for example, in a particular kind of genetic algorithmic method, there are lots of nuances in how you set them up.

You can have for example parallel objectives that are trying to resolve rather than trying to create each ‘individual’ perfectly. Basically you end up with a more negotiable outcome. And it’s a very common way of doing it these days, the process is ‘solving’ for multiple performance goals that are often competing – like getting something that is incredibly light but also incredibly strong. For example if you use a multi objective Genetic Algorithm, you might try to keep an entire set of all solutions or configurations, as we would call them, that you create through all of the generations of the process. The scientific language for this is ‘population’. That’s how you have to talk. You might say, “I have a population of fifty generations of the algorithm. Five thousand individuals would be created throughout the whole process. And in each generation you’re only ‘breeding’ or combining a certain set and you discard the others.” You leave them behind, that’s quite common. And we had a long talk about whether or not we should keep all of the things that were created and the discussion was going on like, “But some of them were just like rubbish. They’re just stupid. We should just kill them, no one needs to see them again.” And I’m like, “well I don’t know, I quite like to keep everybody!”

Of course all you’re really doing is optimizing, tending towards something that’s better and you lose the possibility of chance and miss something. There’s a massive bit of randomness in it and you have a whole set of controls about how much randomness you allow through the generational processes and so I have this massive metaphor and it comes with huge, problematic language around genetics and all that kind of stuff that is encoded with even more problematic language, without any criticality, into the algorithmic process. And then someone is telling me “I’ve got a slider that says increase the randomness on that”. So it’s full of all those things, which I find very challenging.

But if you ever could strip away from the language and all of the kinds of problems, you look at it in purely just what it does, it’s still interesting as a process and it can be useful, but the problem is not what the algorithm does. It’s what culturally those algorithms have come to represent in people’s imagination.

The Hairy Hominid effect

PB: We would like to bring up the Truthful Hairy Hominid here[4]. The figure emerged when we made an excursion to the design department in the basement of the Natural Science Museum in Brussels. They were using a combination of modeling software to update the representation of human species for the ‘Gallery of Humankind’. They were working on one concrete specimen and the designer was modeling their hair, that was then going to be placed on the skin. And someone in our group asked the designer, “How do you know when to stop? How many hairs do you put on that face, on that body?” And then the designer explained that there’s a scientific committee of the museum that handed him some books, that had some information that was scientifically verified, but that all the rest was basically an invention. So he said that it’s more or less this amount of hair or this color, this density of hair. And this is what we kept with us: When this representation is finished, when the model is done and brought from the basement to the gallery of the museum, that representation becomes the evidence of truth, of scientific truth.

PL: It acts like as a stabilization of all of those thoughts, scientific or not, and by making it in that way, it formalizes them and becomes unchallengeable.

PB: The sudden arrival of an invented representation of hominids on the floor of a natural science museum, this functional invention, this efficacy, is turned into scientific truth. This is what we call The Hairy Hominid effect. Maybe you have some stories related to this effect, on the intervention of what counts or what is accountable, what counts? Or is the tunnel already one?

PL: Well, the technology of the tunnel project maybe is, and how we’re using this stuff.

A point-cloud contains millions and millions of data-points from surveys, like in lidar scannning, it’s still really novel to use them in our industry, even though the technology has been around for years.[5] I would say the reason it still gets pushed as a thing is because it has become a massive market for proprietary software companies who say: “Hey, look, you have this really cool map, this really cool point cloud, wouldn’t it be cool if you could actually use it in a meaningful way?” And everybody goes, “yes!”, because these files are each four and a half gigabytes, you need a three thousand pound laptop to open it and it’s not even a mesh, you can’t really do anything with it, it just looks kind of cool. So the software companies go: “Don’t worry about it. We’ll sell you thousands of pounds worth of software, which will process this for you into a mesh”. But no one really is thinking about, well… how do you really process that?

A point-cloud is just as a collection of random points. You can understand that it is a tunnel or a school or a church by looking at it, but when you try and get in there and measure, if you’re really trying to measure a point cloud ... what point do you choose to measure? And whilst they say the precision is like plus or minus zero point five millimeters... well, if that was true, why have we got so much noise?

The only thing that’s real are the data points

PL: One of the things that everybody that everybody thinks is useful, is to do object classification on a point-cloud, to find out what’s a pipe, what’s a light, what’s a desk, what’s a chair. To isolate only those points that you see and then put them on a separate layer in the model and isolate all those things by category. The way that that’s mostly done right now, even in expensive proprietary software, is manually. So somebody sits there and puts a digital lasso around a bunch of points. But then how many of the points, when did you stop, how did you choose how to stop? Imagine, processing ten kilometer of tunnel manually...

PB: It’s nicer to go around with spray paint then.

PL: Definitely.

The most extensive object classification techniques come from autonomous vehicles now, that’s the biggest thing. These data-sets are very commonly shared and they do enough to say, “This is probably a car or this is a sign that probably says this” but everything is guesswork. Just because a computer can just about recognize some things, it is not vision. I always think that computer vision is the wrong term. They should have called it computer perception.

There is a conflation between the uses of computer perception for object classification, around what even is an object and anyway, who really cares whether it’s this type or this, what’s it all for? Conflating object classification with point-cloud technology as a supposedly perfect representation, is actually useless because you can’t identify the objects that you need and anyway, it has all these gaps, because it can’t see through things and then there is a series of methods, to turn that into truth, you de-noise by only sampling one in every three points… You do all of these things to turn it into something that is ‘true’. That’s really what it is like. It’s a conflation of what’s real while the only thing that’s real are the data points, because well, it did capture those points.

A potential for possibilities

PB: When we spoke in Toronto six years ago, you defended generative procedures against parametric approaches, which disguise the probable as possible. Did something change in your relation to the generative and it’s potentially transformative potential?

PL: I think it became more complex for me, when you actually have to do it in real life. I still think that there’s huge risks in both approaches and at the time I probably thought that the reward is not worth the risk in parametric approaches. If you can be good at the generative thing, that’s riskier, it’s much easier to be bad at it, but the potential for possibilities is much higher.

What is more clear now is that these are general processes that you have to encounter, because everybody else is doing it, the bad ones are doing it. And I think it’s a territory that I’m not prepared to give up, that I don’t want to encounter these topics on their terms. I don’t consider the manifestations, those that we don’t like in lots of different ways, to be the only way to use this technology. I don’t consider them to be the intellectual owners of it either. I am not prepared to walk away from these techniques. I want to challenge what they are in some way.

Over the last few years of building things for people, and working with clients, and having to build while we were also trying to build a group of people to work together, you realize that the parametric or procedural approaches give you an opportunity to focus on what is necessary, to clarify the decision-making in all these choices you make. It is more useful in that sense. I was probably quite surprised how little people really wanted to think about those things in generative processes. So we had to start a little bit more simple.

You have to really think first of all, what is it you’re going to make? Is it okay to make it? There’s a lower limit almost of what’s okay to make in a parametric tool because changes are really hard, because you lock in so many rules and relationships. The model can be just as complicated in a generative process, but you need to have a kind of fixed idea of what the relationships represent within your model, within your process. Whereas in generative processes, because of the very nature of the levels of abstraction, which cause problems, there are also opportunities. So without changing the code, you can just say, well, this thing actually is talking about something completely different. If you understand the maths of it, you can assign a different name to that variable in your own mind, right? You don’t even need to change the code.

With a parametric approach you’re never going to get out of the fact that it is about a building of a certain type, you can never escape that. And we built parametric tools to design housing schemes or schools as well as some other infrastructure things, data centers even, and that is kind of okay, because the rules are not controversial when you think about schools for example. And you’re probably thinking, hang on a minute, Phil, these can be controversial, but in the context of our problem definition, they were unchallengeable by anybody, they came from the government.

Showing the real consequences

PL: Parametric approaches make problems in the rules and processes visible. I think that’s a huge thing. Because of the kind of projects we are building, we are given a very hard set of rules that no one is allowed to challenge. So you try and encode them into a parametric system and it won’t work basically.

In the transport infrastructure projects we were doing, there are rule changes with safety, like distance between certain things. And we could show what the real consequences would be. And that this was not going to achieve the kind of safety outcome that they were looking for. Sometimes you’re just making it very clear what it was that they thought that they were asking for. You told us to do it like this, this is what it gives you, I don’t think that’s what you intended.

We never allowed the computer to solve those problems in any of the things we’ve built. It just tells you, just so you know, that did not work, that option. And that’s very controversial, people often don’t really like that. They’re always asking us to constrain them. “Why do you let the system make a mistake?”

Sometimes it is not better than nothing

PB: When we speak to people that work with volumetric systems, whether on the level of large scale databases for plants, or for making biomedical systems … when we push back on their assumption that this is reality, they will say, “Of course the point-cloud is not a reality. Of course the algorithm cannot represent population or desire.” But then when the system needs to work, it is apparently easy to let go of what that means. The need to make it work, erases the possibility for critique.

PL: One of the common responses I see is something like, “Yeah, but it is better than nothing.” Or that is at least part of the story. They have a very Modernist idea that you run this linear trajectory towards complete know-how of knowledge or whatever and that these systems are incomplete rather than imperfect and that if you have a bit more time, you’ll get there. But where we are now, it’s still better than then. So why not use it?

In the construction sector you constantly encounter these unlucky wanna be Silicon Valley tech billionaires, who will just say like, “But you just do it with a computer. Just do it with an algorithm!” They’ve fallen for that capitalist idea that technology will always work in the end. It must work. And whenever I present my work in conferences, I always talk about my team, what people are in the team, how we built it in some way. To the point that actually lot’s of people get bored of it. Other people when they talk about these kinds of techniques will say “We’ve got this bright kid he’s got a PhD from wherever. He’s brilliant. He just sits in the corner. He’s just brilliant.” And of course, it’s always a guy as well. They instrumentalize these people, as the device to execute their dream, which is that the computer will do everything. There’s still this kind of a massively Modernist idea that it’s just a matter of time until we get to that.

Sometimes a point-cloud is not better than nothing because it gives you a whole other problem to deal with, another idea of reality to process. And by the time you get into something that’s usable, it has tricked you into thinking that it’s real. And that’s true about the algorithms as well. You’re wrestling with very complicated processes and by the time you think that you kind of control it, it just controlled you, it made you change your idea of the problem. You simplify your own problem in order that you can have a process act on it. And if you’re not conscious about how you’re simplifying your problem in order to allow these things to act on it, if you’re not transparent about that, if you don’t acknowledge it, then you have a very difficult relationship with your work.

Supposed scientific reality

PL: We use genetic algorithm on a couple of projects now and the client in one project was just not interested in what methods we were using. They did not want us to tell hem, they did not care. They wanted us to show what it does and then talk about that, which is kind of okay. It’s anyway, not their job. The second client was absolutely not like that at all, they were looking for a full explanation of everything that we did. And our explanation did not satisfy them because it didn’t fit with their dream of what a genetic process does.

We were fighting this perception that as soon as you use this technique, why doesn’t it work out of the box? And then we’re building this thing over a matter of weeks and it was super impressive how far we got, but he still told us, I don’t understand why this isn’t finished. It took the US military 50 years to make any of this. Give me a break!

PB: The tale of genetics comes with its own promise, the promise of a closed circuit. I don’t know if you follow any of the critiques on genetics from microchimerism or epigenetics, basically anything that brings complexity. They ask: what are the material conditions in which that process actually takes place? It’s of course never going to work perfectly.

PL: The myth-making comes with the weight of all other kinds of science and therefore implies that this thing should work. Neural networks have this as well, because of, again, this storytelling about the science of it and I think the challenge for those generative processes is exactly in their link to supposed scientific realities and the sort of one-to-one mapping between incomplete science, or unsatisfactory science, into another incomplete unsatisfactorily discipline, without question. You can end up in pretty spooky place with something like a Genetic Algorithms that are abstracted from biochemistry, arguing in a sort of eugenic way.

You can only build one building

PL: I think inherent in all of this science is the idea that there is a right answer, a singular right answer. I think that’s what optimization means. For the sort of stuff we build, we never say, “This is the best way of doing it.” The last mile of the process has to be a human that either finishes it, fills in the gaps or chooses from the selection that is provided. We never ever give one answer.

I think someone in my world would say, “But Phil, we’re trying to build a building, so obviously we can only build one of them?” This is not quite what I mean, I think there’s an idea within all of the scientific constructs of the second half of the twentieth century where computer vision and perception, computer intelligence, whatever you want to call it, and genetics, they’re the two biggest things. Within both of those fields, there’s the idea that we will know, that we will at some point find out a truth about ourselves as humans and about ourselves plus machines. And we will make machines that look like us and then tell ourselves that because the machine performs like this, we are like those machines. I think it’s a tendency which is just super Modernist.

They want a laser line to get to the best answer, the right answer. But in order to get to that, the thing that troubles me probably most of all, and this is true in all of these systems whether parametric or genetic, is the way in which the system assumes a degree of homogeneity.

It does not really matter that it is ultimately constrained

PL: I think with these generative algorithmic processes, people don’t accept constraint either discursively or even scientifically. At most they would talk about the moment of constraint being beyond the horizon of usefulness. At some point, it doesn’t create every possible combination. Lots of people think that it can create every option that you could ever think of. Other people would say that it is not infinite, but it goes beyond the boundary of what you would call, ‘the useful extent of your solution space’, which is the kind of terminology they use. I think that there’s a myth that exists, that through a generative process, you can have whatever you want. And I have been in meetings where we showed clients something that we’ve done and they say, “Oh, so you just generated all possible options.” But that’s not quite what we did last week!

There’s still that sort of myth-making around genetic algorithms, there’s an illusion there. And I think there’s a refusal to acknowledge that the boundary of that solution space is set not really by the process of generation. It’s set at the beginning, by the way in which you define the stuff that you act on, through your algorithmic process. I think that’s true of parametrics as well, it’s just that it’s more obviously to improve metrics. Like, here’s a thing that affects this thing. And whether you complexify the relationships between those parameters, it doesn’t really matter, it’s still kind of conceptually very easy to understand. No matter how complex you make the relations between those parameters, you can still get your head around it. Whereas the generative process is a black box to a certain extent, no one really knows, and the constraint is always going to be on the horizon of useful possibilities. So it doesn’t really matter that it is ultimately constrained.

We’re not behaving like trained software developers

PL: By now we have about twenty people on our team and they’re almost all architects.

When I do a presentation in a professional context, I have a slide that says, “We’re not software developers, but we do make software.” And then I try to talk about how the fact that we’re not trained as software developers, means that we think about things in different ways. We don’t behave like them. We don’t have these normative behaviors from software engineering either in terms of what we create or in the way in which we create things. And as we grow, we make more things that you could describe as software, rather than toolkits or workflows.

After one of these events, someone came up to me and said, “Thank you, that was a very interesting talk. And then she asked, “So who does your software development? To who do you outsource the development?” It is completely alien to this person that our industry could be responsible for the creation of software itself. We are merely the recipients of product satisfaction.

Architects are not learning enough about computation technology either practically or critically, because we’ve been kind of infantilized to be the recipient discipline.

Not everyone can take part

PB: We noticed a redistribution of responsibilities and also a redistribution of subjectivity, which seems to be reduced to a binary choice between either developer or a user.

PL: I think that’s true. It feels like we’re back in the early nineties actually. When computational technology emerged into everyday life, it was largely unknown or was unknowable at the time to the receiving audience, to the consumers. There was a separation, an us and them, and even talking about a terrible version of Windows or Word or something, people were still understanding it as something that came from this place over there. Over the last two or three decades, those two things are brought together, and it feels much more horizontal; anybody can be a programmer. And now we’re back at the place where we realise that not everybody can be part of creating these things at all.

Governments have this idea that we’ll all be programmers at some point. But no, we won’t, that’s absolutely not true! Not everybody’s going to learn. So one of the things I try to hold on specifically is that we need to bring computational technology to our industry, rather than have it created by somebody else and then imposed on us.

The goal is not to learn how to be all a software company or a tech company.

If something will work, why not use it?

PB: We are troubled by the way 3D techniques and technologies travel from one discipline to another. It feels almost impossible to stop and ask “hey, what decisions are being made here?” So we wanted to ask you about your experience with the intense circulation of knowledge, techniques, devices and tools in volumetric practice.

PL: It is something that I see every day, in our industry, and in our practice. We have quite a few arguments about the use of image recognition or facial recognition technologies for example.

When technologies translate into another discipline, into another job almost, you don’t just lose the ability to critique it, but it actually enhances its status by that move. When you reuse some existing technology, people think you must be so clever to re-apply it in a new context. In the UK there are tons of examples of R&D government funding that would encourage you to use established existing techniques and technologies from other sectors and reapply them in design and construction. They don’t want you to reinvent something and they certainly don’t want you to challenge anything. You’re part of the process of validating it and you’re validated by it. And similarly, the people of the originating discipline get to say, “Look how widely used the thing we created is”, and then it becomes a reinforcement of those disciplines. I think that it’s a huge problem for anyone’s ability to build critical practices towards these technologies.

That moment of transition from one field to another creates the magic, right? A technology apparently appears out of nowhere, lands fully formed almost without friction and also without history. It lacks history in the academic sense, the scientific process and indeed, also lacks the labor of all of the bodies, the people that it took to make it, no one cares anymore at that point.

What I’ve seen in the last five years is that proprietary software companies are pushing things like face recognition and object classification into Graphical User Interfaces (GUI’s), into desktop software. Something like a GAN or whatever is not a button and not a product; it is a TensorFlow routine or a bunch of python scripts that you get off GitHub.

There’s a myth-making around this, that makes you feel like you’re still engaged in the kind of practice of creating the technique. But you’re not, you’re just consuming it. It’s ready-made there for you. Because it sits on GitHub, you feel like a real coder, right? I think the recipient context becomes infantilized because you’re not encouraged to actually create it yourself.

You’re presented with something that will work, so why not use it? But this means you also consume all of their thinking all of their ways of looking at the world.


The first part of the conversation which took place on May 4 2015, Flight Cafe East, Toronto. Original recording: flightcafe.MP3

Comprehensive Features (a conversation with Phil Langley)

The environment of simulation

PB: There seem to be a few problems with how 3D operates. The first is how volumetric representations of physical objects in digital space are constrained by the the mesh, a very particular way of dealing with inside and outside. The second is related to resolution: a disconnect between the hyper-real and a crude underlying structure. And the third is related to the parametric, and the limited 'space of possibilities' it creates.

PL: I think these issues all intersect as well. There is also something about the environment of simulation, the world in which the simulation is created in order to then simulate within, or to represent within or to modify within.

From my background as an architect you have the simulation of a buildings' performance, that is the way I am normally exposed to it so: how much energy would it use, would it be structurally stable, these kinds of questions. In order to simulate that, you would have a digital model and you run a bunch of algorithms over it that would test solar gain (?), structural stability, things like this. But that digital model is very much a reduction of the most detailed model that might exist during the design process, partly due to computing power, or mostly due to it in fact, you have to simplify the geometry of that building down to boxes, effectively, and that reduces the relationship between two rooms that share a wall and the algorithm does not really care exactly how big that room is, it can't really change with a change in geometry, even in the order of a couple of meters in floor area, it will not really effect the simulation outcome. The simulation I am talking about is of a certain type (?), the constructing of a world in which you test your 'proposal', and very often that is merely talked about as the environment in which you construct, and in a structural simulation it is what kind of algorithms are you using, is it efficient, does it take this into consideration, does it take that into consideration, but very rarely do you see what you need to do to the geometry in order to expose or subject it to the algorithm.

I think there is an almost blindness in the understanding that the nature of the algorithm effects the nature of the model. So in architectural discourse on digital modeling at the moment you have a lot of hyperbole about how this stuff gets way more capable of modeling huge amounts of detail, the model is almost quasi one-to-one, it might have every nut and bolt, the window ledge will be modeled with the exact profile... There is a fallacy held that that geometry is then translated in the environmental simulation, and will render a better result but it doesn't. The model that you see on your screen is not the model that is actually analysed.

A symbiotic relationship

PB: When we try to critique the crudeness of 3D models, the response is often: “we need more data”, “if only we had more computing power”, “it is a question of efficiency” … Is that the kind of reduction you talk about, meaning is it a problem that can be solved if you would have more data-points or rendering power, so that the reduction could be minimized?

PL: My understanding of this, is that there is a symbiotic relationship between the algorithm that runs the simulation and the structure of that algorithm. It is not that it could ever understand the profile of a window sill or an opening or an overhang. It has been designed with the computing power available in mind to deal with the fact that you just can't … So, there is a link between the digital environment in which you are simulating, the processes you modify, you have to modify the model, the physical materiality of computing itself. You can't just swap one in and one out. They have been involved symbiotically probably.

I use the word 'reduction' and it is pejorative often and I don't necessarily mind it but what I mind is the lack of visibility in that process.

The social condition of use

PL: I think the bigger issue is more how that materiality is encoded into the model, what you consider worthy of encoding and what is left out. So the materiality of a building might get reduced to a face, a single line of a plane in the model, so that wall that should be 300 mm thick only needs to be described as a single face for the purposes of the energy use. But there is no user in there. There are no people. There is no behavioural model, the materiality of us is left out of that simulation. There is the tick, cross ... tick, cross next to the things you are going to include in it, and across the spectrum of super curvy algorithmic architecture, and super boxy algorithmic architecture, it is still the same question for me actually: what have you included and what have you left out in order to get to this outcome?

PB: Do you think that there is a way that 'thinking with computers' could be more interesting than ticking boxes?

PL: Well, in the nineties there was much excitement about the potential of all these new software techniques in architecture, also linked to post-modernism, as architects were as always late to the party, and the potential of that kind of expressive geometry, that you could generate more easily, using those kind of algorithmic techniques, to be a way for example for allowing for a redundancy of functional spaces, so rather than saying: "this zone is for this, and this is for this, enclosed by walls, and whatever" you would actually have a place of encounter, in which uses and behaviour that you would not have predicted could flourish somehow. You are not so prescriptive anymore for what kinds of spaces you make. It was a very exciting moment. When you look at the designs, fifteen years later, and the glossy pictures by people like NOX for example, it looks a bit naive at best and suspicious at worst. But they come from this tradition, of being less prescriptive and less deterministic about behaviour inside a building. That was a very exciting moment, but I feel they bankrupted themselves, almost literally actually, by trying to build these things. The problem for me is ... it is not just the materiality of the building that creates the social condition of its use.

Even if you are trying to encourage more different types of sociality in your building, you probably shouldn't be doing that only through the materiality of the building.

And that is where it hit its buffer, I would say. And that mode ended. They are still doing the same thing, regardless of what it looks like or the technique ... or what they hope the space provides. They still use the same technique as the Greeks were doing by building a big temple and going ...

PB: "This is where you go!"

PL: Exactly. So it wasn't really plugging into other kinds of ... What groups like Stealth did, although they left the digital behind a little bit now.

PB: When looking at them now, it is hard to imagine how these iterations could be more interesting, more inclusive somehow and talk back to multiple parameters, not just those provided by the software.

PL: This is how I got interested in coding at all. I was fascinated by these glossy images of almost impossible geometry and the 'coolness'. I wanted to be able to make something like that - not necessarily to build it, but to somehow produce it on the screen. But I never really did it in the end. By the time I learned how to code my interest in that approach had ended and although I wanted to make shapes like that, I didn't want buildings to be like that. In fact, it was more the potential for constant iteration and change that interested me and that possibility of not just that thing we have seen a million times before being produced and almost 'de-professionalising' the process somehow, from an architectural point of view. In the professional world there are countless books about what you do with concrete, what you do with bricks, what you do with steel, what you do with timber whereas these other techniques were saying 'actually, who knows what you do with this!', and that was quite exciting. There was this instability around what the profession was about and somehow the existing processes of design were questioned by it.

PB: But then in that de-professionalisation, how does the user find a place?

PL: Well, in that model in the 1990s, not all! It is conditioned by the buildings' materiality as its main approach and for a long time I found it very difficult to think of a way that it could be more than this. And actually the trick is not to be obsessed only with the materiality and to broaden what that 'expressive' model is, what it could be based on and including more things inside the model. And i think it is very hard to do! But it comes down to the way in which you are defining that system in which you are operating in. The architectural paradigm is a Newtonian physics world of gravity. Actually that is a bit harsh, it is a bit more sophisticated than that, but it still is Newtonian in the user experience even if the underlying physics is a bit more sophisticated than that. It is a world in which there are no people... let alone different types of people. There are only these conditions of performance, there is only this materiality, there is only this physical sciences behaviour. But why couldn't you build a different world? And then you could test other things. Of course, now you have software that can test how people move in an airport, but that is not really ...

That's a really bad way of doing it and people like Space Syntax, for example are involved in that kind of thing and it is something that I completely don't agree with. It is the reduction of the user to generalised behaviour.

Limited materiality

PB: You were talking about your thesis and that in the chapter about simulation, you wanted to look at MakeHuman. So this software - well I'm not sure it is about people! - but at least it is about body shapes. Can you explain why, as someone looking at let's say algorithmic architecture and the digital technologies around it, a software for building humanoid figures would be considered interesting? How does it relate? Or not?

PL: I think it relates very, very closely firstly because of this question of materially. Somehow the software is deciding what is going to be used to define a human and that materiality is defined through the mesh but it doesn't include anything beyond the surface, apart from the topological skeleton which everything hangs off. But it is a very limited materiality that they have defined and that is exactly what an architectural model would do. So it is fixing that but also giving you some feeling of flexibility which, again, is what this kind of parametric architecture does. There is this sense that you are somehow in control of potential outputs or outcomes and it is not an uncommon thing to see in a digital architectural design process. Certainly, it is a the very least implicit that you fix some things - of the design brief let's say - and then you are able to flex, but only within those things and who determines where those things sit and who fixes them? this is a very common approach, in digital and non-digital architectural design processes and the Make Human software has exactly this kind of interface in terms of the parametric sliders, in terms of the visualisation change in the color or the materiality of the body itself. And also what it is leaving out - there is no nervous system, there is no circulatory system (such as you can describe those two things) and certainly no personality!

I guess the analogy between architecture and Make Human would be less strong between architecture and Make Human if the features hadn't been so 'comprehensive'. Because it tries to do so much, it makes it really similar to architectural software, it is also really trying to give you everything. But it just can't, so why try?

Inappropriate features

PB: So in MakeHuman there is the discreteness of the sliders in terms of how they are presented and how they are actually not separate. There is the fact that they are aligned as similar horizontalities and at the same scale, which is really nauseating. Then there is the different types of binary that are going on, no?

Man/Woman is quite a different horizon than Young/Old! And the most surprising one then, is to put race in this. Have you found something in the documentation, or the way in which the code is written?

PL: I have been trying to go through it to find when it appeared. So one of things I am interested in this kind of software is how stable it becomes, in terms of its features and functionality. At what point can you no longer change the software and you can only talk about what it is good and bad for. In my attempts to modify the source code, my technical ability was nowhere near enough to deal with the maths behind the mesh management. It is way beyond my coding skills and maths. But I was able to understand the parametric links and to de-couple some of them or to alter the proportionality in order to generate some other results and I was able to change the color effect of the 'ethnicity' sliders to RGB. So there is a huge amount of stability in it now that to change it yourself you would need to be so skilled. It is so tightly wound, it is so efficient in what it does that to unpick it was way beyond me. Instead, I tried to go back to find out when the features arrive, at what point. Having looked at a bunch of their repositories of previous versions, the 'ethnicity' is there for ages. It is not a thing that comes late, it is in their minds it seems a long, long time ago. If you look on the message boards you can see a few people complain and then they get told to shut up!

PB: Complain about what?

PL: About the 'ethnicity' feature being inappropriate, but there didn't seem to be many people that supported that view.

Mesh integrity

PL: If you go into the folder structure and see how it is organised you can see the 'genitals' folder or whatever. But you can't just swap in an alternative model. There is an individual body part model - say the genitals or the eyes - but you can't just take that file and swap it for your own one. It has to fit a certain file type and be processed into a certain type of data structure for it to be readable. I never found an easy way of just swapping the parts. And again, I sort of understand the huge amount of technical knowledge that has gone into making this very efficient system, but it is super frustrating not to be able to drop things in and out, even if it made an imperfect mesh. And their whole thing is about making the perfect mesh actually, whereas I would rather be able to put a head on the end of a leg and see what it looks like! It is really un-playful in that way.

PB: From what we saw at GenderBlending I am confused, especially when we were making gender changes. Looking at it with Xavier, who has an understanding of what it means to change the gender of a physical body, he insisted that it is not like sticking on genitals. On the one hand, it feels like a fragmented body image, and you describe that the elements are in different places, and at the same time it is quite unwilling to let go of it's illusion of 'wholeness'.

PL: Absolutely. I think that the fragmented nature I could deal with. If it was a collaging tool almost, that wouldn't bother me at all. It is not a direct analogy to actual bodies and would not be misunderstood as that. But actually it is a kind of collaging tool, hidden behind a huge amount of complex maths and code, to produce the impression of biological integrity. That goes with the 'scientific' data from which the body parts are generated and the supposed precision of those sliders so that fact that it is trying to create the impression that it is a useful representation of bodily process and change. Actually, it is not at all. It is a collage of things, which I would be much more comfortable with because it is more obviously wrong, and it would not be misunderstood as anything other. And it would be far more fun!

PB: The first thing you want to do is make a 'collage body' when you think what it would mean to make a humanoid in software. And then you realise it is the hardest thing. Students managed to glitch it, and then they could find ways to expose it. For example, if you make certain combinations of 'race' and 'gender' then the skin colour doesn't extend to the genitals and they start to stick out because they are coloured differently. So in that sense you can start to reveal the fragmentation that is in there but that you cannot work with. So I was trying to talk about the three interconnected problems that we see coming together. The collage character that is hidden behind the need for digital integrity that becomes confused with the image of a natural body.

PL: Yes, that kind of mesh integrity...

PB: This is where the mesh problem and the resolution problem start to meet each other.

PL: That mesh integrity is driven by the underlying topology of the 'skeleton' and the resolution of that skeleton is super scary because you have these two oppositional things. A complete reduction in the freedom movement though the simplification of the skeleton and a mesh that supposes to include all details and wrinkles.

An acknowledged relationship

PL: OK. So, in a generative technique - genetic algorithm, neural network, cellular automata would be described as generative techniques - you establish a set of rules that are explored through the execution of the algorithm, the outcome of which is determined by a process whose level of complexity means that you could not have predicted the result. So there is a gap between the system you put in place and the thing it produced. Whereas in a parametric system you make a box and you play within the box. In a parametric system you are rarely surprised.

PB: Because you get what you want.

PL: Right, you have already set a boundary on that space of possibility. And in fact, one of the images I did for GenderBlending was to put all of the sliders of MakeHuman to the left and then all of the sliders to the right - I know this was a little facetious and it is not really how the software is structured - but that's it, it's going to be within those two bounds.

Whereas in generative approaches, the idea is that you don't quite know what it will produce and then in some way the algorithm has an agency, which of course you are part of. For me it is a far nicer way of working, but again, so long as it is an acknowledged relationship between the agencies.

PB: So, you say that 'generative' has a complexity that might surprise you, is then the moment that agency goes away the moment I understand what is happening? Is the complexity pseudo-magic? Is it just because I don't understand that I am surprised?

PL: No, I don't think it is that you don't understand. It is about the 'predictability' rather than 'understanding'. So if you have a genetic algorithm, which is a nice example, you can 'evolve' a design solution. Your algorithm becomes a metaphor of evolutionary process, at least as it was understood 20 years ago because the algorithm is always super-far behind! which I don't really mind as it is an analogy more than anything else (at least it should be taken as that). So, you evolve a design solution and you are having a symbiotic relationship with...er...I am trying to use the term 'digital companion' in my writing.

It is not pseudo magic

I don't want it to be misunderstood as that thing that happens at the moment where everyone goes SARCASTICALLY 'aren't smart-phones changing us' or 'isn't digital technology...' or 'oooo my phone is really clever'. That's not what I mean. It is more that you make it and remake you relationship and it evolves and you evolve. So, with a genetic algorithm, you made a 'thing' from yourself and you can be very open and honest about that. It is a way of expanding that space of possibilities for me. It is, of course, limited. It is not magic, it is not pseudo magic - it can't just CLICK create something. It is unpredictable, but it is not that it can make anything at all - there are still boundaries to what it is capable of producing it is just that you can't follow in your mind each one of those steps.

I think that at least you can't see the boundaries, that is what I would be interested in, that they are outside the periphery of your vision. That would be the ideal of a really good generative algorithm, from my point of view. That is not how everybody else would necessarily see it, some people really want to, you know...erm...they want their genetic algorithm to solve a very specific problem. Wheras, for me, parametric modeling technology and techniques and interfaces and whatever, puts 'front and centre' that it is in this sand pit. Whatever you are going to get, you can see the boundaries within which it is going to come.

PB: You could also say that's 'visibility'?

PL: Yes, absolutely. But I think on your understanding of the process. It could be very explicit - 'it's just this. this is a thing that does that'. In MakeHuman, they don't say 'this is just some of the things that exist in the world'. It is very much 'this is the range of your humanity'!

I think there are also some very practical things that could be done. I think some of it is about it's aspiration to represent 'all' physical bodies - i think they could be a bit more explicit about how it doesn't.

I think that the interface itself could be much more...er...honest about interaction between variables.

Everything is parametric

PL: I think that 'parametric' is a much abused term now. Everything is parametric. As soon as you start to codify somehow, everything has a parameter. In architecture, there's lots of, I would say - and you can quote me on this - odious books by people like Patrick Schumacher about how parametric architecture is a new 'thing', which I just can't deal with. Everything is parametric. A door is a parametric - it can be a bit wider a bit taller, that's parametric. As soon as you have a variable it is parametric, so i don't see it in that way at all.

PL: I see it more about ... ER ... the difference being that the use of parametric ... or the appropriation of parametric as a term ... is a way of making a toy to play with. The sliders, make it reconfigurable but there are somehow a limited amount of possible outcomes. So you could calculate every single position of slider, every single possible variation of that.

PB: So there is a number, even if there are a lot of variations.

PL: Yes, but you could still calculate it - that's a number right? With a genetic algorithm, yes, you can do it, but that number is much much bigger.

When you have a parametric system, you can basically know everything it can make as soon as you have defined it, every single possibility already exists.

So there are things that you did not really think that were necessarily possible and this is a very directed example, you know you are trying to evolve a brandy glass, you know what a that looks like, as a human you have a relationship to what it is producing. But when you change a variable in that system, to the topological definition of the glass, the degree of freedom you give, you know like you define this by point point point, sweep ... so than it can start to do almost anything. So as soon as you change some of these definitions, you really change huge amounts of what it can do, which is never true in a parametric system.

The visibility question ... how do you convey that complex process to somebody ... an interface is always about hiding the complex process somehow, could you ever make an interface that allowed you to get what is going on?

Interfacing the possible

PB: It is hard to interface something like that. You talked about that earlier, the need for interfaces in parametric software, that it does not really exist without it. In these generative processes, the exploration itself, the probing of what is possible, is where the interface seems to be?

PL: One of the reasons you don't see very many, very good graphical user interfaces on genetic algorithm software, because it is not very helpful. Because in the end, what you are changing is so much under the hood ... There is no benefit in a slick, or even a contained graphical user interface. You change something so fundamental, when you modify it.

That is what I like, it is one of the things I try to talk about when I talk to students about code, is that stability of the software becomes problematic. The interface marks a level of stability ...

Well, this is now a system

PB: We talked about disconnected skins and crude skeletons, and how this limits humanoid figures, but also interactions between 'people' seems to be missing.

PL: I think the nervous system in general is a thing that is missing from often. One of the things I put in the notes for Topological Subjectivities at GenderBlending, is systems in bodies, an interesting one for me, because it is all about the line, the boundary you put around a bunch of things, and say, "well, this is now a system". With the circulatory system or the whatever system, and the nervous system is one where that boundary is felt very fixed in Western science for a long time, and now it is much more fuzzy and people are a lot less clear about where it begins and where it ends.

Everyone used to think it was the brain, and now they go ... oh, the spinal chord is kind of interesting too, so it is probably a bit of that. And there is a much more radical working field (?) where your entire nervous system contains all your brain power. You can't have brain power anymore, it is what some radical thinkers say, it is not a way (another way?) to talk about our capacity. Our capability ... and you can imagine the historic change from thinking the heart was the only thing that mattered. What I like about that, is that it is not that our bodies have changed, or have particularly evolved during that period, the materiality is exactly the same. But the topological understanding is what is evolving. And that is I think again the stability issue. During GenderBlending at some point we were talking about species, categorization, all this kind of stuff. For me the problem is not necessarily categorization, but more the stability of categorization, the fixity of it.

Categorization for me is a normal thing to do, it is just horrible when you fix it, "this is the only way you can be called". Topology in math, like set theory, is all about an approach that objects can be in multiple sets. It can be more things to other objects. It is one property that puts it in other sets, although 'property' is not a very good word. But it does not preclude it from appearing in another set.


Notes

  1. See: Possible Bodies, “MakeHuman,” in this same book. https://possiblebodies.constantvzw.org/book/index.php?title=MakeHuman
  2. See: “Comprehensive Features, a conversation with Phil Langley,” on-line. https://possiblebodies.constantvzw.org/book/index.php?title=Interview_with_Phil_Langley
  3. TensorFlow is “An end-to-end open source machine learning platform” used for both research and production at Google. https://www.tensorflow.org/
  4. Item 086: The Trutful Hairy Hominid https://possiblebodies.constantvzw.org/inventory/?086
  5. Lidar is an acronym of “light detection and ranging” or “laser imaging, detection, and ranging”.