This is a bit of a complex topic sitting in my brain. Hopefully it will make sense on this first pass.
The topic is the process of writing itself. The topic is integration. This is one are where I never see any discussion.
"Integration," as I use it here, is the non-linear process of taking many elements and turning them into a coherent whole, a finished and imperfect piece.
Integration has multiple aspects, which I will simply here into inputs, outputs, and conflicts. Inputs include all the requirements of a work, along with everything else that comes along in the creator's mind. The output is a flawed piece of work. The conflicts are those flaws that exist and must be address for the work to be a successful piece of work.
Sounds dry, doesn't it? Yet, in those few sentences exist the resolution to all the literary/artistic advice in the world. Should you write to market? Should you make the work sexier? Should you revise the dialog? Should you deviate from the accepted market norms? How far? What structure?
The first thing to understand about integration is that the process is not linear. There's a reason that we don't have computers telling stories yet. The creation of a sentence both requires huge amounts of context and changes contexts at the same time. Every sentence changes the story, changes our understanding of the story, and changes the potential end of the story. As a writer works, the beginning, end, and middle exist in a relationship with each other, even if those pieces haven't been written yet.
The second part of integration is that integration takes from multiple elements, including itself. Which elements? Anything in or potentially part of the human experience, which is a stunning array of possible elements. The writer can't possibly include all elements, so the author must choose which elements to include, that choosing being part of the non-linear process, and that choosing influencing the other elements that are included.
The result of all this integration is a flawed product. The product must be flawed, for everything real is flawed. In order for the work to reach maximum effectiveness, the work must actually be perfected in one area to the detriment of others. Not all flaws are equal. The most critical flaws must be addressed while the less critical flaws must be let be.
So, if we are good, rule-obeying writers that mind our mentors and use all our best skills, we will produce something that has flaws. In order to fix those flaws, we must therefore break some rule that we learned or go against the advice of our mentors. Likewise, we may have followed no rules, creating a work with many flaws, but where it matters, our writing works despite our flaws. Because this work is so crazily rigged, any attempt to fix it properly would destroy the parts that work, because the parts that work depend on the parts that don't work.
The final part of integration is creating a coherent whole, a story that feels like a story rather than a series of vaguely related sentences with no notion between them. This whole is contained in no single sentence of the story, yet anyone who reads it should get the same sense that the story forms a whole unit.
With as complicated as integration seems, its a wonder that we can write at all, which is just about right. Writing is cognitively complicated, open-ended, and iterative. Producing a good writer ought to be hard, just as training any artist ought to be hard. The subject matter is non-trivial while the modes of expression are many.
The first part of integration is creating limits. You can't have everything, so having something immediately creates a limit. Picking a genre creates a limit. Creating a focus creates a limit. Creating limits is what allows you to discard most possibilities.
The second part of integration is imperfection. Since perfection is impossible, you must make active choices about what's important, about which parts need to work. This is why there are no rules in writing. In fact, there are rules, but as creating introduces limits, so too creating requires eliminating the rules that introduce the largest problems. Your goal is not to create a perfect embodiment of the rules, your goal is to create the best story that you can. This is why good writers say, "there are no rules, only suggestions." A working story trumps theory every time. Theory may help you to produce a working story, but it's the considered breaking of theory that makes a story work.
So when the best writers tell you to write, that they're telling you is to practice your integration. That's all internal work. The only way to learn it is to do it. As you do it, you learn to break the rules in a particular way, one that takes your own idiosyncrasies into account. This is your voice. This is what makes you YOU and not somebody else.
Integration. It's the bicycle riding of skills. Once you get it, you get it, but there's always more there to get. You never stop changing, so integration is a skill that always changes with you.