As part of the termination, Allied World will receive a termination fee of $35 million plus $13.3 million of merger-related expenses. Transatlantic will also be obligated to pay Allied World an additional $66.7 million if it enters into or recommends a competing transaction within the next year.
"Although disappointed we were unable to complete the proposed merger with Transatlantic, Allied World's core business strategy remains intact," said Allied World Chief Executive Scott Carmilani.
In commenting on the move Friday, Transatlantic also announced that current Chief Operating Officer Michael C. Sapnar would step into the chief executive officer role effective Jan. 1. Current CEO Robert F. Orlich, whose retirement was previously announced, will remain on the board of directors.
The all-stock deal would have exchanged 0.88 of an Allied World share, valued at about $47.05 based on Thursday's closing price, for each Transatlantic share.
Approval of the deal had begun to look increasingly unlikely, after three shareholder advisory firms urged Transatlantic investors to reject the merger and Transatlantic's largest investor said it was voting against it.
Transatlantic said the company's largest investor, Davis Selected Advisors, offered support for the termination of the deal Friday. Davis held a roughly 24% stake in the company as of late last month.
"Davis Advisors applauds Transatlantic's efforts to create value for shareholders with an intelligent capital management plan while at the same time remaining open to other strategic alternatives," the firm said in a statement.
Standard & Poor's earlier this week said it was no longer considering an upgrade of the credit ratings of Transatlantic and Allied World as there was less than a 50% chance that Transatlantic shareholders would approve the merger.
Transatlantic on Friday also estimated that third-quarter per-share earnings will range between 85 cents and $1.15, below the $1.25 profit projected by analysts surveyed by Thomson Reuters. The company said that on a preliminary, pretax basis, net catastrophe costs were between $55 million to $65 million. The total includes costs related to storms in Denmark and Hurricane Irene.
Transatlantic's board increased the company's stock buyback authorization by $455 million, bringing the total allowed under the plan to $600 million.
The company expects to complete the buybacks in 2012, with half planned for the rest of this year.
Insurance and reinsurance company Validus Holdings Ltd. has also been angling for Transatlantic. Earlier this week, it launched an effort to replace the company's board of directors. Validus in July took a $3.5 billion tender offer directly to Transatlantic's shareholders after the two companies failed to agree on provisions in a confidentiality agreement that would have allowed negotiations to progress.
As a result of the termination of the deal, Transatlantic said it canceled a special meeting for shareholders called for next week.