It's always fun to see if the locations used in the video can be pinpointed. With "The Night We Called It a Day,", the opening scene was shot in front of Màs Malo, an east-LA-style Chicano resturant located at 515 W. 7th Street.
Mr. D walks past the Màs Malo storefront to the door you can see in the alley on the right. The neon "Club" sign shown in the video apparently came from the prop department.
A post in the EDLIS Café Facebook page relates that the other notable location for the video shoot was The Ardmore Wilshire at 722 Ardmore Avenue in Korea Town Los Angeles on January 12th of this year.
A member of the café posted the color photo below of the building's atrium, which was featured in a wide-angle shot in the video.
Frank rehearsed constantly for the next three weeks, afternoons at the Palladium nightclub, in Los Angeles, before Tommy showed up, just Frank and Lyle (Skitch) Henderson or Joey Bushkin on the piano behind him, in the quiet stillness of the huge, empty dance hall. He knew exactly which songs he wanted to wax. They were all ballads, naturally, all dripping with romance: there was “The Night We Called It a Day,” by these new kids Matt Dennis and Tom Adair, who’d written “Let’s Get Away from It All” and “Violets for Your Furs.” There was a sweet Hoagy Carmichael number that hadn’t been recorded much, “The Lamplighter’s Serenade.” And then two classics: Kern and Hammerstein’s “The Song Is You” and Cole Porter’s equally immortal “Night and Day,” whose lyrics he’d once blown in front of the great man himself when Porter stopped by the Rustic Cabin...
The recording session, which took place at RCA’s Los Angeles studios on the afternoon of Monday, January 19, 1942, went off perfectly. Pointedly, there was no drummer. Nor did Dorsey attend the session—both of the singles that resulted (released on RCA’s discount Bluebird label) were credited to Frank Sinatra with “Orchestra conducted by Axel Stordahl.” ~ Frank: The Voice