set automatically allocates new column slots if needed

[ben-schwen]

Closes #1831
Closes #4100
Supersedes #5269

Doesn't this open the same can of worms that we're seeing with setDT()? #6706, #6723, ...

We need to consider set(env$nm, ...) and a whole litany of other ways set() can be used on set(<call>, ...) besides as set(<name>, ...), right?

Originally posted by @MichaelChirico in #5269 (comment)

Basically yes, but we have to carry this ball anyway because of #7500

[github-actions]

• HEAD=set_automate_overalloc stopped early for DT[,.SD] improved in #4501
  

Generated via commit 21a3567

Download link for the artifact containing the test results: ↓ atime-results.zip

Task	Duration
R setup and installing dependencies	2 minutes and 45 seconds
Installing different package versions	43 seconds
Running and plotting the test cases	4 minutes and 21 seconds

[codecov]

Codecov Report

❌ Patch coverage is 66.66667% with 2 lines in your changes missing coverage. Please review.
✅ Project coverage is 98.96%. Comparing base (dc69b3d) to head (21a3567).
⚠️ Report is 1 commits behind head on master.

Files with missing lines	Patch %	Lines
R/data.table.R	66.66%	2 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #7538      +/-   ##
==========================================
- Coverage   98.97%   98.96%   -0.02%     
==========================================
  Files          87       87              
  Lines       16733    16737       +4     
==========================================
+ Hits        16561    16563       +2     
- Misses        172      174       +2     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[MichaelChirico]

I think we need ; DT and ; DT2 here, a for loop returns NULL so now the test just checks the code runs:

x = for (i in 1:10) i
dput(x)
# NULL

[MichaelChirico]

too many parentheses :)

I propose this helper:

.set_needs_alloccol = function(x, value) {
  if (truelength(x) <= length(x)) return(TRUE)
  if (selfrefok(x, verbose=FALSE) >= 1L) return(FALSE)
  if (is.null(value)) return(TRUE)
  if (!is.list(value)) return(FALSE)
  any(vapply_1b(value, is.null))
}