Avery was born in New York, in 1885 and in 1898 moved with his family to Wilson Station, Conn. From 1901 to 1911 he held many mechanical and construction jobs, but he became interested in art while taking a lettering course (some time between 1905 and 1911) at the Connecticut League of Art Students in Hartford. Avery continued classes there until 1918, when he entered Hartford's School of the Art Society. He moved to New York in 1925 and in 1926 married Sally Michel, a fellow artist who was often the subject of his work. Avery attended evening classes at the Art Students League and in 1927 started exhibiting regularly in group shows. The following year Avery and artist Mark Rothko became friends, and Rothko, in turn, introduced him to Adolph Gottlieb and Barnett Newman, all leading Abstract Expressionist artists. His friendship with them did not lead him to share their commitment to total abstraction, however.
In 1932 the Avery's daughter, March, was born, and she appears frequently in her father’s paintings. From 1935 on, Avery was represented by a series of top galleries in New York. He spent the summers of 1957–1960 in Provincetown, Mass., the scene of many of his paintings.Although rooted in the American Scene tradition, Avery's work was too abstract to assign him a place in that group; and though he was a friend of the foremost Abstract Expressionists, his work was too representational to belong to the non-objective movements of the 1940s and 1950s. Avery's colorful, simplified forms, and explicit yet subtly toned contours, defy classification. Yet the freshness and uncomplicated nature of his images linked him with other independent American modernists, such as Arthur Dove, John Marin, and Lee Gatch, all, like Avery, artists whose work was greatly admired and collected in depth by Duncan Phillips.