The similar cleanup with a lot of other changes were committed
to the main branch. Is easier to revert this change, copy the
scripts from main, and merge things back.
This reverts commit 4bc1ba3c2d.
The `make update` will now always update all initialized submodules,
including the library ones.
The `make_update.py --no-libraries` will neither initialize nor update
pre-compiled libraries or tests.
Pull Request: https://projects.blender.org/blender/blender/pulls/118812
First time builders get spooked by this warning and start
installing python trying to resolve it. This isn't needed
and we should only warn about python being missing if the
lib folder is cloned, but python is still somehow not found.
This was "fixed" a few times before, but it took a bit
to find the right filename to check for the new library
folder.
Solves the issue of the script potentially sitting for a long time
without having any progress reported. Confusingly, such behavior
depends on Git version.
In older versions (< 2.33) there will be progress reported, but it
then got changed by the commit in Git:
https://github.com/git/git/commit/7a132c628e57b9bceeb88832ea051395c0637b16
The delayed checkout is exactly how Git LFS integrates into the Git
process. Another affecting factor for the behavior is that submodule
configured to use "checkout" update policy is forced to have quite
flag passed to the "git checkout":
https://github.com/git/git/blob/v2.43.2/builtin/submodule--helper.c#L2258
This is done to avoid the long message at the end of checkout about
the detached state of HEAD, with instructions how to resolve that.
There are two possible solutions: either use "rebase" update policy
for submodules, or skip Git LFS download during the submodule update.
Changing the update policy is possible, but it needs to be done with
a bit of care, and possible revised process for updating/merging
tests data.
This change follows the second idea of delaying LFS download for a
later step, so the process is the following:
- Run `git submodule update`, but tell Git LFS to not resolve the links
by using GIT_LFS_SKIP_SMUDGE=1 environment variable.
- Run `git lfs pull` for the submodule, to resolve the links.
Doing so bypasses hardcoded silencing in the Git. It also potentially
allows to recover from an aborted download process.
The `git lfs pull` seems to be a nominal step to resolve the LFS links
after the smudging has been skipped. It is also how in earlier Git
versions some Windows limitations were bypassed:
https://www.mankier.com/7/git-lfs-faq
The submodule update now also receives the "--progress" flag, which
logs the initial Git repository checkout process, which further
improves the feedback.
The byproduct of this change is that an error during precompiled
libraries and tests data update is not considered to be fatal.
It seems to be more fitting with other update steps, and allows
more easily reuse some code.
There is also a cosmetic change: the messages at the end of the
update process now have their own header, allowing more easily
see them in the wall-of-text.
Pull Request: https://projects.blender.org/blender/blender/pulls/118673
Likewise, skip tests update when --use-tests is not provided.
It was a bit of ambiguous situation because libraries and tests
are technically submodules. After some feedback it seems that it
is better to ignore submodule for libraries and tests unless
requested explicitly.
Pull Request: https://projects.blender.org/blender/blender/pulls/118631
git on windows is giving no status updates while git lfs
is downloading the libs, causing people to assume its
stuck and interrupt the download. This change adds a
rather large visual message telling people not to do
that.
also it was not respecting people selecting n on the lib
download so some extra messaging has been added there
as well.
Git escaped a quote and used all of the cli arguments as the
folder name.
Also fixes:
- Progress on the initial clone so it doesn't appear stuck
- re-detect python after updating libs, so you don't have to run
make update twice on a fresh clone.
- Hide python not found warning unless the lib folder exists (this
seemingly got lost in one of the merges)
If LFS hooks are installed manually the automatic installation would
fail.
This change makes it so `lfs` is a valid command of `git`, ensuring
that the package is installed. If the installation fails assume it
is due to tricky local setup, and do not fail.
Pull Request: https://projects.blender.org/blender/blender/pulls/118618
- git lfs install was called a bit too late, after the libraries
has been checked out, leaving the checkout in a non-resolved
state.
- Update blender code first, allowing proper submodule hash to
be pulled.
Pull Request: https://projects.blender.org/blender/blender/pulls/118615
This change makes it so build system and update utilities for Blender builds
are using pre-compiled libraries and other resources attached as Git modules
instead of using checkout of SVN repositories in the parent folder.
The directory layout:
```
* release/datafiles/
* assets/ -> blender-assets.git
* publish/
* ...
* README.txt
* lib/
* darwin_x64/ -> lib-darwin_x64.git
* darwin_arm64/ -> lib-darwin_arm64.git
* linux_x64/ -> lib-linux_x64.git
* windows_x64/ -> lib-windows_x64.git
* tests/
* data/ -> blender-test-data.git
```
The changes about configuring the actual Git sub-modules are not included
into this patch, as those require repository to actually exist before it
can be used.
The assets submodule is enabled by default, and the rest of them are
disabled. This means that if someone runs `git submodule update --init`
they will not get heavy libraries. The platform-specific and tests
related submodules are enabled when using `make update` or `make test`.
All the submodules are tracked: this means that when new commits are
done to the submodule, the blender.git repository is to be updated to
point them to the new hash. This causes some extra manual work, but it
allows to more easily update Blender and its dependencies to known good
state when performing operations like bisect.
Ref #108978
Pull Request: https://projects.blender.org/blender/blender/pulls/117946
Main improvements:
* Spatial structure (Kd-tree) build is now multithreaded.
* Kd-tree switched to use cache-friendlier TreeLets.
* Field fixed some non-deterministic behavior when spatial cache does
not receive any training data during a training iteration due to a
large number of training iterations.
* Fixed build problems on (non-Mac) ARM systems.
Pull Request: https://projects.blender.org/blender/blender/pulls/118328
This was already the minimum requirement for Intel and Apple Silicon
GPUs. It is required for the Metal backend to work correctly.
Previously the minimum for AMD GPUs was 10.15.
Pull Request: https://projects.blender.org/blender/blender/pulls/118287
Turns out we were not building OSL with OptiX enabled anymore.
Also check now if the OSL builds has OptiX support and if not
disable it in Cycles.
Building OSL with support for this (still) does not require
either the OptiX SDK or CUDA, it only needs LLVM.
Pull Request: https://projects.blender.org/blender/blender/pulls/118234
- Commands which have arguments split over multiple lines use
indented lines.
- Wrap lines where multiple commands run using "&&".
- Blank lines between multiple commands helps the text from becoming
too dense.
This is not a hard requirement to be able to build the libraries, but
these versions should be written down somewhere. So compare compiler
versions as part of make deps setup.
Pull Request: https://projects.blender.org/blender/blender/pulls/117457
Flex's bundled configure depended on aclocal-1.15 which has been
updated to 1.16.
Resolve by regenerating configure files on Linux which in turn adds a
new dependency on texinfo.
Flex's bundled configure depended on aclocal-1.15 which has been
updated to 1.16.
Resolve by regenerating configure files on Linux which in turn adds a
new dependency on texinfo.
Campbell Barton