Directionally everything rings true to my work experience, but I'm getting hung up on a lot of details (especially on software architect as a particular person's role and its interaction with other business and technical roles). I've more often worked in contexts where architecture in the sense you're describing it is the output of interaction among (1) various technical leads/managers with project delivery responsibility and (2) product managers/owners/designers who act as proxies for the business and users. I think 2 is playing the role of what you are referring to as "the marketing department", but IME experience the marketing department is usually playing the different role of figuring out how to push already-made product or identify trends/opportunities.
This only matters substantively because architectural responsibilities are often layered and situational: as a front-end lead I may be handed technical component or business domain architectures in some contexts and hand them out in others. More usually, though, I am contributing to meetings where those architectures are being defined interactively (before then being handed off to IC engineers).
> IME experience the marketing department is usually playing the different role of figuring out how to push already-made product or identify trends/opportunities.
I will correct this. It depends on the industry and the particular company, probably. What I wrote is accurate for the companies I have had good looks into, but they were all SMEs at the interface between biotech and info tech, which is not representative.
The inbound vs outbound marketing distinction is relevant here. Inbound includes "finding out what software people are willing to pay for," which can include requirements analysis, which overlaps into architecting in some cases.
> the output of interaction among (1) various technical leads/managers with project delivery responsibility and (2) product managers/owners/designers
Yes; in a product-oriented company, the product managers/designers are often in the marketing department. Or, that was my experience in a particular industry thirty years ago! I don't think it's atypical now either, but it's certainly not universal. E.g., unlikely in the case of developing a system for internal use only.
> software architect as a particular person's role
> Architecting is a task, not necessarily a job. Depending on the size of a project and a company’s management practices, it can be the responsibility of different job roles. Some projects have a person specifically designated as the software architect, which might or might not be their full-time job, and might or might not be their job title. There may be a team of several people whose full-time responsibility is architecting a single large project. On a small project, it may be no one’s assigned responsibility. In that case, it usually gets done by the most senior technical person on the project.
I wonder if part of the problem is that software projects are often part of a rationalizing project to begin with. For example, "digital transformation" in business often requires that existing business processes be broken down and reformatted into things that can be managed by software. This might be driven by higher ups who want a more legible / measurable / manageable process, so the rational perspective can be baked into the project from the outset.
I love the concepts here and yet I wonder if it could 'pop' more. The most successful parts, for me, have real examples, like the discussion of Agile. I wonder if you could introduce parts with more case examples or even dialogues / characters like in 'Meaningness and Time'?
Thanks for subscribing, and thanks for the suggestion!
My original intention was for the entire software section to be written in the style of “Getcha Boots On!” (https://meaningness.substack.com/p/getcha-boots-on), which I extracted from an early version.
That would be much more fun! But also much more work to write, and I want to get this finished someday (this year I hope). Also, it would be much more to read.
Also, this is meant to be “a textbook of meta-rationality.” Textbooks just are dry; it’s the nature of the genre, for good reasons.
Serializing a textbook as Substack posts is a dubious proposition—but since the textbook is my main writing project currently, that’s what’s going to happen.
However, I have changed my mind about concentrating on just that. Tentatively I intend to alternate posts: half from the meta-rationality book, and half drawn from my various other projects. Which I also want to progress!
Yes, this seems right. It is interesting how requirements evolve dynamically,
Hypothetical example, any resemblance to any real project entirely co-ncidental (*cough* CHERI *cough*) "CPU design is to be compatible with version III of the MIPS instructions set architecture. Any additional features will need justification," ok, so now you've you've got chip designers writing hardware description language, compiler writers writing compiler backends, operating systesm guys doing a port of the machine dependent part of BSD... Some time later, when you have something at least roughly working ... "It turns out that that it really kills performance if you don't have this particular feature that was added in version IV of MIPS ... see rationale document and simulated measurements," Update the spec ... hardware design guys compiler guys, operating systems guys work in the feature, Some time, possibly a lot later, "Actually, it turns out that every single user-space feature of MIPS version IV is essential if you don]t want to kill performance, see the attached rationale documents and simulated benchmarks."
A: "Ok, so as an external formal methods consultant I've been looking at your HOL4 security proof, and I can't help noticing there's an instance of undefined behavior here."
B: "it's ok. Only privileged operating system code can put the chip into that state, and the operating system is Trusted to Not Do That, See this section here in the Architecture document."
A: "Yeah, but what actually happnes?"
B: "Chip catches fire possibly."
A: "What? Surely you are joking, or being metaphorical."
B: "The content addressable memory in the translation lookaside buffer is put into an inconsistent state, possible causing a large flow of electrical current, which puts the whole chip out of spec for how much heat it can dissipate... Probably wont happen with our design. Other implementations might, in principle, have that issue,"
A: "Well, that's not very satisfactory,, it it?
B: "OS is trusted code"
A: "But, if e.g. an attacker has obtained root access..."
B: "Well, yes, you could defend against it in hardware. But is would be annoying,"
(B signs and spends the rest of the day writing an architecture change)
I swear that that part of the MIPS Instruction Set Architecture has got be one of the most ... surprising ... things I have ever read. Take a look at the sample code for initializing the Translation Lookaside Buffer.
As a software developer and architect, this section is easy to read, enjoyable, and quite correct relative to my experience. (Caveat: Ignacio seems more correct about the role/dept norms this decade. Marketing is all outbound, inbound has become biz-dev somewhat but very largely PMs, in part because of the rationalized agile-industrial complex eating everything.)
So this is a "thumbs up" from me.
I see non-software people asking for more examples here.... And it makes me imagine an interleaving of fiction and non-fiction (ala G.E.B.!?) chapters because realistic examples are so costly in word-count they're hard to weave into textbook non-fiction.
Thanks! Your comment and Ignacio's made me wonder where in the reporting structure product managers go now, so I did a little web reading. Several sources confirmed that it used to be in marketing, but then you got people without technical knowledge making technical decisions, so some companies moved them into the IT department, but then the PMs didn't get enough customer contact, and requirements analysis got done de facto by Sales which was terrible, so the hip thing now is to have a Chief Product Officer who reports directly to the CEO and PMs report up to them. Is that what you have seen?
(Apparently, from what I've read, this isn't consistent and everyone is pretty confused about where PMs should go, and different companies still have them in marketing or tech or sales).
Yes a CPO and their small PM org is what I'm most familiar with, but most common for the less hip may just be "Engineering" which is not IT (at least for any company that has primary emphasis on making a software product). PMs, Devs, Test Eng is a common 3-way reporting split within Engineering/Development. Or, in agile land, primary reporting may be team-based multidisciplinary with only dotted-line connection to discipline.
Separate question: I wonder if software architecting as defined here has overlap with areas such as systems engineering, a supposed interdisciplinary field
I know little about systems engineering. I read the first few paragraphs of the wiki article (https://en.wikipedia.org/wiki/Systems_engineering) and there does seem to be an overlap. Software architecting is more specific, I think; it’s specifically software, and it covers just the high-level design for a software system.
In the absence of a broader meta-rational culture, that is infeasible for most participants. Managers are unwilling to give up predictability and control; technical people are unwilling to improvise in a volatile, nebulous context, rather than methodically Solving a fixed Problem.
1. I wonder what a meta rational culture looks like
2. I wonder if all successful meta rational culture look the same or different
> There is no overall plan for anything. No one even knows what the organization is supposed to be trying to do. It’s completely unclear who is responsible for what. People frequently take on tasks they have no qualifications for, just because they feel like it. No one understands the work they are doing, and no one thinks that’s a problem. There are frequent arguments about what to do, with no procedure for coming to a decision, and no one has the authority to conclude one.
I wonder if on hindsight, there's **always** a way to craft a narrative on an apparent overall plan for such a workplace.
Maybe some of the most successful organizations are like that. Where all the books written about them are all narratives on hindsight, but actually they are all unplanned (to some extent)
> I wonder if on hindsight, there's **always** a way to craft a narrative on an apparent overall plan for such a workplace.
I think that may be true... one use of plans-as-communication (discussed in "What are plans for?", https://sci-hub.se/https://linkinghub.elsevier.com/retrieve/pii/S0921889005800260 ) is to make sense of what you have done retrospectively—as a rational reconstruction—which may help you figure out where you went right or wrong, and do better in similar situations in the future.
> Maybe some of the most successful organizations are like that. Where all the books written about them are all narratives on hindsight, but actually they are all unplanned (to some extent)
I think this is also true. As "What are plans for?" says, a plan is rarely/never a fully detailed description of what is to be done that you follow mindlessly. It's a sketch map that you improvise with the aid of.
I agree with the history of how things went with Agile - that’s how I see it too! However, these things are described in rather broad terms and I didn’t see any of the promised anecdotes?
I suspect that some real-world examples would be helpful to readers who haven’t worked as software engineers.
I think the published version may have been revised from the informal memo version in the linked documents, so I'm somewhat reluctant to update the link. It's probably on Sci-Hub?
Directionally everything rings true to my work experience, but I'm getting hung up on a lot of details (especially on software architect as a particular person's role and its interaction with other business and technical roles). I've more often worked in contexts where architecture in the sense you're describing it is the output of interaction among (1) various technical leads/managers with project delivery responsibility and (2) product managers/owners/designers who act as proxies for the business and users. I think 2 is playing the role of what you are referring to as "the marketing department", but IME experience the marketing department is usually playing the different role of figuring out how to push already-made product or identify trends/opportunities.
This only matters substantively because architectural responsibilities are often layered and situational: as a front-end lead I may be handed technical component or business domain architectures in some contexts and hand them out in others. More usually, though, I am contributing to meetings where those architectures are being defined interactively (before then being handed off to IC engineers).
Thanks, this is useful!
> IME experience the marketing department is usually playing the different role of figuring out how to push already-made product or identify trends/opportunities.
I will correct this. It depends on the industry and the particular company, probably. What I wrote is accurate for the companies I have had good looks into, but they were all SMEs at the interface between biotech and info tech, which is not representative.
The inbound vs outbound marketing distinction is relevant here. Inbound includes "finding out what software people are willing to pay for," which can include requirements analysis, which overlaps into architecting in some cases.
> the output of interaction among (1) various technical leads/managers with project delivery responsibility and (2) product managers/owners/designers
Yes; in a product-oriented company, the product managers/designers are often in the marketing department. Or, that was my experience in a particular industry thirty years ago! I don't think it's atypical now either, but it's certainly not universal. E.g., unlikely in the case of developing a system for internal use only.
> software architect as a particular person's role
In the previous episode (https://meaningness.substack.com/i/144307788/what-is-software-architecting), I wrote:
> Architecting is a task, not necessarily a job. Depending on the size of a project and a company’s management practices, it can be the responsibility of different job roles. Some projects have a person specifically designated as the software architect, which might or might not be their full-time job, and might or might not be their job title. There may be a team of several people whose full-time responsibility is architecting a single large project. On a small project, it may be no one’s assigned responsibility. In that case, it usually gets done by the most senior technical person on the project.
I wonder if part of the problem is that software projects are often part of a rationalizing project to begin with. For example, "digital transformation" in business often requires that existing business processes be broken down and reformatted into things that can be managed by software. This might be driven by higher ups who want a more legible / measurable / manageable process, so the rational perspective can be baked into the project from the outset.
Yes, that seems right!
I love the concepts here and yet I wonder if it could 'pop' more. The most successful parts, for me, have real examples, like the discussion of Agile. I wonder if you could introduce parts with more case examples or even dialogues / characters like in 'Meaningness and Time'?
Thanks for subscribing, and thanks for the suggestion!
My original intention was for the entire software section to be written in the style of “Getcha Boots On!” (https://meaningness.substack.com/p/getcha-boots-on), which I extracted from an early version.
That would be much more fun! But also much more work to write, and I want to get this finished someday (this year I hope). Also, it would be much more to read.
Also, this is meant to be “a textbook of meta-rationality.” Textbooks just are dry; it’s the nature of the genre, for good reasons.
Serializing a textbook as Substack posts is a dubious proposition—but since the textbook is my main writing project currently, that’s what’s going to happen.
However, I have changed my mind about concentrating on just that. Tentatively I intend to alternate posts: half from the meta-rationality book, and half drawn from my various other projects. Which I also want to progress!
Yes, this seems right. It is interesting how requirements evolve dynamically,
Hypothetical example, any resemblance to any real project entirely co-ncidental (*cough* CHERI *cough*) "CPU design is to be compatible with version III of the MIPS instructions set architecture. Any additional features will need justification," ok, so now you've you've got chip designers writing hardware description language, compiler writers writing compiler backends, operating systesm guys doing a port of the machine dependent part of BSD... Some time later, when you have something at least roughly working ... "It turns out that that it really kills performance if you don't have this particular feature that was added in version IV of MIPS ... see rationale document and simulated measurements," Update the spec ... hardware design guys compiler guys, operating systems guys work in the feature, Some time, possibly a lot later, "Actually, it turns out that every single user-space feature of MIPS version IV is essential if you don]t want to kill performance, see the attached rationale documents and simulated benchmarks."
I dont know if I dare say this one....
A: "Ok, so as an external formal methods consultant I've been looking at your HOL4 security proof, and I can't help noticing there's an instance of undefined behavior here."
B: "it's ok. Only privileged operating system code can put the chip into that state, and the operating system is Trusted to Not Do That, See this section here in the Architecture document."
A: "Yeah, but what actually happnes?"
B: "Chip catches fire possibly."
A: "What? Surely you are joking, or being metaphorical."
B: "The content addressable memory in the translation lookaside buffer is put into an inconsistent state, possible causing a large flow of electrical current, which puts the whole chip out of spec for how much heat it can dissipate... Probably wont happen with our design. Other implementations might, in principle, have that issue,"
A: "Well, that's not very satisfactory,, it it?
B: "OS is trusted code"
A: "But, if e.g. an attacker has obtained root access..."
B: "Well, yes, you could defend against it in hardware. But is would be annoying,"
(B signs and spends the rest of the day writing an architecture change)
EEEEEEEEEEEEKKK!
I swear that that part of the MIPS Instruction Set Architecture has got be one of the most ... surprising ... things I have ever read. Take a look at the sample code for initializing the Translation Lookaside Buffer.
Thanks! Much more on nebulous requirements in the next installment.
The CHERI project is really interesting to me. (I used Multics back in the stone age...) I hope it's going OK!
As a software developer and architect, this section is easy to read, enjoyable, and quite correct relative to my experience. (Caveat: Ignacio seems more correct about the role/dept norms this decade. Marketing is all outbound, inbound has become biz-dev somewhat but very largely PMs, in part because of the rationalized agile-industrial complex eating everything.)
So this is a "thumbs up" from me.
I see non-software people asking for more examples here.... And it makes me imagine an interleaving of fiction and non-fiction (ala G.E.B.!?) chapters because realistic examples are so costly in word-count they're hard to weave into textbook non-fiction.
Thanks! Your comment and Ignacio's made me wonder where in the reporting structure product managers go now, so I did a little web reading. Several sources confirmed that it used to be in marketing, but then you got people without technical knowledge making technical decisions, so some companies moved them into the IT department, but then the PMs didn't get enough customer contact, and requirements analysis got done de facto by Sales which was terrible, so the hip thing now is to have a Chief Product Officer who reports directly to the CEO and PMs report up to them. Is that what you have seen?
(Apparently, from what I've read, this isn't consistent and everyone is pretty confused about where PMs should go, and different companies still have them in marketing or tech or sales).
Yes a CPO and their small PM org is what I'm most familiar with, but most common for the less hip may just be "Engineering" which is not IT (at least for any company that has primary emphasis on making a software product). PMs, Devs, Test Eng is a common 3-way reporting split within Engineering/Development. Or, in agile land, primary reporting may be team-based multidisciplinary with only dotted-line connection to discipline.
Separate question: I wonder if software architecting as defined here has overlap with areas such as systems engineering, a supposed interdisciplinary field
I know little about systems engineering. I read the first few paragraphs of the wiki article (https://en.wikipedia.org/wiki/Systems_engineering) and there does seem to be an overlap. Software architecting is more specific, I think; it’s specifically software, and it covers just the high-level design for a software system.
In the absence of a broader meta-rational culture, that is infeasible for most participants. Managers are unwilling to give up predictability and control; technical people are unwilling to improvise in a volatile, nebulous context, rather than methodically Solving a fixed Problem.
1. I wonder what a meta rational culture looks like
2. I wonder if all successful meta rational culture look the same or different
> I wonder what a meta rational culture looks like
Well, I described one imaginary one here: https://metarationality.com/meta-rational-workplace
Some R&D teams (in both academia and the private sector) get pretty close to this. It's rare.
> I wonder if all successful meta rational culture look the same or different
I wonder too. I would guess the answer is "same in some ways, different in others."
I re-read the meta rational workplace
and i quote the first part
Here’s a description of a utopian workplace:
> There is no overall plan for anything. No one even knows what the organization is supposed to be trying to do. It’s completely unclear who is responsible for what. People frequently take on tasks they have no qualifications for, just because they feel like it. No one understands the work they are doing, and no one thinks that’s a problem. There are frequent arguments about what to do, with no procedure for coming to a decision, and no one has the authority to conclude one.
I wonder if on hindsight, there's **always** a way to craft a narrative on an apparent overall plan for such a workplace.
Maybe some of the most successful organizations are like that. Where all the books written about them are all narratives on hindsight, but actually they are all unplanned (to some extent)
> I wonder if on hindsight, there's **always** a way to craft a narrative on an apparent overall plan for such a workplace.
I think that may be true... one use of plans-as-communication (discussed in "What are plans for?", https://sci-hub.se/https://linkinghub.elsevier.com/retrieve/pii/S0921889005800260 ) is to make sense of what you have done retrospectively—as a rational reconstruction—which may help you figure out where you went right or wrong, and do better in similar situations in the future.
> Maybe some of the most successful organizations are like that. Where all the books written about them are all narratives on hindsight, but actually they are all unplanned (to some extent)
I think this is also true. As "What are plans for?" says, a plan is rarely/never a fully detailed description of what is to be done that you follow mindlessly. It's a sketch map that you improvise with the aid of.
I agree with the history of how things went with Agile - that’s how I see it too! However, these things are described in rather broad terms and I didn’t see any of the promised anecdotes?
I suspect that some real-world examples would be helpful to readers who haven’t worked as software engineers.
> I didn’t see any of the promised anecdotes?
I couldn't figure out what you meant, and then now I'm guessing you misread "antidotes" in the subtitle as "anecdotes"?
> I suspect that some real-world examples would be helpful to readers who haven’t worked as software engineers.
Yes, thanks, that's probably right. There will be more detailed (although made-up) examples in the next section.
Oops! Yeah, I misread.
Excellent article! Small typo: “quantitatively and quantitatively”?
Thank you very much, on both counts! I've fixed the typo now :)
Some links for your intriguing 'what are plans for?' paper which include full text (and, I think, are even legal)....
https://apps.dtic.mil/sti/tr/pdf/ADA200726.pdf (authentic document scan)
https://dspace.mit.edu/handle/1721.1/6487 (postscript file, slightly tidier though still looks like a scan - my adobe pro manages to open it)
Thanks!
I think the published version may have been revised from the informal memo version in the linked documents, so I'm somewhat reluctant to update the link. It's probably on Sci-Hub?
ah yes
https://sci-hub.se/https://linkinghub.elsevier.com/retrieve/pii/S0921889005800260
hadn't realised the different status of the versions!
Thanks!
I’m not sure if they’re actually different. It was a long time ago.