Man I take a couple of days off and you guys go hog wild in here.
WRT post Midway and the IJN, Shattered Sword has a good amount of detail about changes for IJN doctrine, they actually DID learn quite a bit but were slow at instituting it. There were changes that were supposed to be in place at the time of I think Eastern Solomons, but were delivered too late to the fleet to act on. They were literally trying to fly out to the carriers with new changes, read Shattered Sword for details.
Also during the Solomon's Campaign, the IJN was still in the game and gave as good as it got, perhaps better if you counter them losing one light carrier and zero fleet carriers vis a vis USN loss of Hornet and Wasp with Saratoga damaged and sent stateside (again). They just didn't have the ability to STAY in the game. Their building program hadn't foreseen the protracted type of war they found themselves not only mired in, but by virtue of the sneak attack on Pearl Harbor, virtually guaranteed they would be in. Mentioned in SS is the fact that the Japanese never understood the war they instigated, but also did not understand their opponent, and even their own goals for going to war with the United States in the first place could be said to be rather nebulous and short sighted.
Much like expecting the American's at Midway to "go lowing to their slaughter" (Shattered Sword) they totally misread the effect on the American psyche regarding the way they started the war. Seen as weak willed for starters I think that America's resolve to prosecute the war to the fullest extent was quite a shock to them, and indeed, I think many still didn't accept that they had their collective asses kicked from hell to breakfast, even in 1945.
They also apparently failed to take into account America's economy, which had been running in low gear due the Great Depression,
but was still in 1940, FIVE TIMES that of Japan's, which was already straining due to their mucking about in China and Southeast Asia etc. So as their economy started reaching its breaking point in 1940-41, America's went from that low gear into overdrive.
But back to Midway, I can't take any credit for what I've written, just paraphrasing from Shattered Sword, which I believe is one of the most comprehensive books written on ANY battle in the history of warfare. There are others AS good, but none better I believe.