It was just a short time before Stevenson was incensed by Joe’s hiring of a carriage, presumably to transport supplies, and fired him. Joe apologized and agreed to submit all costs before they were incurred. Stevenson took him back and they sailed. By September, Stevenson had fired him again, this time because Joe had argued with him about the use of flash powder to take a photograph at night. As they sailed for Samoa, Joe became deathly ill with influenza and was sent on to his wife in Sydney. Stevenson believed Joe would die.
When they all ended up in Sydney, Stevenson became incensed again at Joe’s purchase of silk handkerchiefs and the fact that he sold a picture and didn’t turn over the money. All of this Joe had done from his “death bed” (as Stevenson described it in one of his letters), fully aware that he had been fired and was living on allowance from Stevenson. Joe’s wife had been told by Stevenson that he would support them until there was a place for them in Samoa, and then he left them there. Unbeknownst to Joe, a good portion of his photographic work for Stevenson was destroyed in a fire aboard the Janet Nicoll which Stevenson and his wife and step-son had taken from Sydney for a further cruise before returning to Samoa.
Here is Joe’s diary: