Commit Graph

104 Commits

Author SHA1 Message Date
534b9b35dd Fix job status aggregation logic (#34823) (#34835)
Backport #34823 by nienjiuntai

Co-authored-by: JIUN-TAI NIEN <44364165+nienjiuntai@users.noreply.github.com>
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
2025-06-24 23:23:16 +08:00
038990e0ff Fix ephemeral runner deletion (#34447) (#34513)
Some checks are pending
release-nightly / nightly-docker-rootless (push) Waiting to run
release-nightly / nightly-binary (push) Waiting to run
release-nightly / nightly-docker-rootful (push) Waiting to run
2025-05-21 18:02:34 -04:00
cdac263bb8 Only git operations should update last changed of a repository (#34388) (#34427)
Some checks are pending
release-nightly / nightly-binary (push) Waiting to run
release-nightly / nightly-docker-rootful (push) Waiting to run
release-nightly / nightly-docker-rootless (push) Waiting to run
Backport #34388 by @lunny

Try to fix #32046

Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
2025-05-11 23:23:12 +02:00
4ed07244b9 actions artifacts api list/download check status upload confirmed (#34273)
Some checks are pending
release-nightly / nightly-binary (push) Waiting to run
release-nightly / nightly-docker-rootful (push) Waiting to run
release-nightly / nightly-docker-rootless (push) Waiting to run
cron-translations / crowdin-pull (push) Has been skipped
* fixes a fixture status to upload confirmed
* add another fixture as noise to break tests as soon they are exposed
to api
* v4 delete test added check that artifact is no longer visible in
internal api with status pending delete
* removal of http 404 on empty list: actions/upload-artifact@v4 now
backoff on http 404 of ListArtifacts endpoint
* fixes artifacts with pending delete etc. are able to be found and
downloaded if the storage is not freed
2025-04-28 16:31:53 +00:00
21b43fce08 Actions Runner rest api (#33873)
Some checks are pending
release-nightly / nightly-binary (push) Waiting to run
release-nightly / nightly-docker-rootful (push) Waiting to run
release-nightly / nightly-docker-rootless (push) Waiting to run
cron-translations / crowdin-pull (push) Has been skipped
Implements runner apis based on
https://docs.github.com/en/rest/actions/self-hosted-runners?apiVersion=2022-11-28#list-self-hosted-runners-for-an-organization

- Add Post endpoints for registration-token, google/go-github revealed
this as problem
  - We should deprecate Get Endpoints, leaving them for compatibility
- Get endpoint of admin has api path /admin/runners/registration-token
that feels wrong, /admin/actions/runners/registration-token seems more
consistent with user/org/repo api
- Get Runner Api
- List Runner Api
- Delete Runner Api

- Tests admin / user / org / repo level endpoints

Related to #33750 (implements point 1 and 2)
Via needs discovered in #32461, this runner api is needed to allow
cleanup of runners that are deallocated without user interaction.

---------

Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
2025-04-18 15:22:41 +00:00
ee3c82f874 Enable addtional linters (#34085)
enable mirror, usestdlibbars and perfsprint 
part of: https://github.com/go-gitea/gitea/issues/34083

---------

Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
2025-04-01 10:14:01 +00:00
0fde8ecd55 Enable testifylint rules (#34075)
Some checks are pending
release-nightly / nightly-binary (push) Waiting to run
release-nightly / nightly-docker-rootful (push) Waiting to run
release-nightly / nightly-docker-rootless (push) Waiting to run
enable testifylint rules disabled in:
https://github.com/go-gitea/gitea/pull/34054
2025-03-31 01:53:48 -04:00
2a59dfbd47 enable staticcheck QFxxxx rules (#34064)
Some checks are pending
release-nightly / nightly-binary (push) Waiting to run
release-nightly / nightly-docker-rootful (push) Waiting to run
release-nightly / nightly-docker-rootless (push) Waiting to run
cron-translations / crowdin-pull (push) Has been skipped
2025-03-29 17:32:28 -04:00
0c6957ef8d Download actions job logs from API (#33858)
Some checks are pending
release-nightly / nightly-binary (push) Waiting to run
release-nightly / nightly-docker-rootful (push) Waiting to run
release-nightly / nightly-docker-rootless (push) Waiting to run
cron-translations / crowdin-pull (push) Has been skipped
Related to #33709, #31416

It's similar with
https://docs.github.com/en/rest/actions/workflow-jobs?apiVersion=2022-11-28#download-job-logs-for-a-workflow-run--code-samples.

This use `job_id` as path parameter which is consistent with Github's
APIs.

---------

Co-authored-by: ChristopherHX <christopher.homberger@web.de>
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
2025-03-26 11:30:52 -07:00
8f051d598c Added Description Field for Secrets and Variables (#33526)
Fixes #33484

---------

Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
2025-03-17 19:24:54 +00:00
65e2411394 Feature: Ephemeral action runners (#33570)
Some checks are pending
release-nightly / nightly-binary (push) Waiting to run
release-nightly / nightly-docker-rootful (push) Waiting to run
release-nightly / nightly-docker-rootless (push) Waiting to run
cron-translations / crowdin-pull (push) Has been skipped
* This includes a runner mock test for hardend PickTask behavior like
described in my proposal
* Runner register ephemeral flag introduced in
https://gitea.com/gitea/act_runner/pulls/649

Closes #32461
2025-03-14 12:27:24 -07:00
a92d5f65ce Fix auto concurrency cancellation skips commit status updates (#33764)
* add missing commit status
* conflicts with concurrency support

Closes #33763

Co-authored-by: Giteabot <teabot@gitea.io>
2025-03-10 15:58:48 -07:00
6c8fb8d455 Small refactor to reduce unnecessary database queries and remove duplicated functions (#33779) 2025-03-04 18:25:20 +00:00
dbed39d632 Add migrations and doctor fixes (#33556)
Fix #33535
2025-03-02 21:01:28 -08:00
ce65613690 Fix Untranslated Text on Actions Page (#33635)
Some checks are pending
release-nightly / nightly-binary (push) Waiting to run
release-nightly / nightly-docker-rootful (push) Waiting to run
release-nightly / nightly-docker-rootless (push) Waiting to run
cron-translations / crowdin-pull (push) Has been skipped
Fix the problem of untranslated text on the actions page

Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
2025-02-18 11:29:08 +00:00
2b8cfb557d Artifacts download api for artifact actions v4 (#33510)
* download endpoint has to use 302 redirect
* fake blob download used if direct download not possible
* downloading v3 artifacts not possible

New repo apis based on GitHub Rest V3
- GET /runs/{run}/artifacts (Cannot use run index of url due to not
being unique)
- GET /artifacts
- GET + DELETE /artifacts/{artifact_id}
- GET /artifacts/{artifact_id}/zip
- (GET /artifacts/{artifact_id}/zip/raw this is a workaround for a http
302 assertion in actions/toolkit)
- api docs removed this is protected by a signed url like the internal
artifacts api and no longer usable with any token or swagger
  - returns http 401 if the signature is invalid
    - or change the artifact id
    - or expired after 1 hour

Closes #33353
Closes #32124

---------

Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
2025-02-16 08:32:54 +08:00
e741448a14 Fix various problems (artifact order, api empty slice, assignee check, fuzzy prompt, mirror proxy, adopt git) (#33569)
* Make artifact list output a stable order
* Fix #33506
* Fix #33521
* Fix #33288
* Fix #33196
* Fix #33561
2025-02-13 03:26:27 +08:00
e9b98aef44 Enhance routers for the Actions runner operations (#33549)
Some checks are pending
release-nightly / nightly-binary (push) Waiting to run
release-nightly / nightly-docker-rootful (push) Waiting to run
release-nightly / nightly-docker-rootless (push) Waiting to run
- Find the runner before deleting
- Move the main logic from `routers/web/repo/setting/runners.go` to
`routers/web/shared/actions/runners.go`.

---------

Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
2025-02-11 01:39:10 +00:00
c422f179dd Enhance routers for the Actions variable operations (#33547)
- Find the variable before updating or deleting
- Move the main logic from `routers/web/repo/setting/variables.go` to
`routers/web/shared/actions/variables.go`.

---------

Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
Co-authored-by: Giteabot <teabot@gitea.io>
2025-02-11 04:44:04 +08:00
2ea929a952 Refactor RefName (#33234)
And fix some FIXMEs
2025-01-13 14:01:53 +08:00
32d45ee069 Update status check for all supported on.pull_request.types in Gitea (#33117)
Some checks are pending
release-nightly / nightly-binary (push) Waiting to run
release-nightly / nightly-docker-rootful (push) Waiting to run
release-nightly / nightly-docker-rootless (push) Waiting to run
Thanks @Zettat123
Follow #33116
Fix #33051

on.pull_request.types doc:

https://docs.github.com/en/actions/using-workflows/events-that-trigger-workflows#pull_request

on.pull_request.types added in this PR:
```
assigned, unassigned, review_requested, review_request_removed, milestoned, demilestoned, labeled, unlabeled
```

unsupported types in Gitea:
```
// Unsupported activity types:
// converted_to_draft, ready_for_review, locked, unlocked, auto_merge_enabled, auto_merge_disabled, enqueued, dequeued
```

TODO:
- [x] add test
2025-01-08 01:58:47 +08:00
9bfa9f450d Refactor "string truncate" (#32984) 2024-12-26 11:56:03 +08:00
7553ae1a57 Use env GITEA_RUNNER_REGISTRATION_TOKEN as global runner token (#32946)
Fix #23703

When Gitea starts, it reads GITEA_RUNNER_REGISTRATION_TOKEN
or GITEA_RUNNER_REGISTRATION_TOKEN_FILE to add registration token.
2024-12-23 09:59:16 +00:00
2d7e6e9482 Fix various trivial problems (#32861)
Some checks are pending
release-nightly / nightly-binary (push) Waiting to run
release-nightly / nightly-docker-rootful (push) Waiting to run
release-nightly / nightly-docker-rootless (push) Waiting to run
cron-translations / crowdin-pull (push) Has been skipped
1. add/improve comments to help future readers could understand the
problem more easily.
2. add an error log to LDAP with username fallback
3. use `or` instead of `Iif` for "repo/branch_dropdown" (`Iif` was a
mistake, but it doesn't really affect the UI)
4. add `tw-font-mono` style to container digest to match dockerhub
5. fix a bug in RepoBranchTagSelector: the form is not updated when
there is no click to an item

---------

Co-authored-by: delvh <dev.lh@web.de>
2024-12-17 01:15:18 +00:00
22c4599542 Improve Actions status aggregations (#32860)
Some checks are pending
release-nightly / nightly-binary (push) Waiting to run
release-nightly / nightly-docker-rootful (push) Waiting to run
release-nightly / nightly-docker-rootless (push) Waiting to run
Make the result the same as GitHub:

* all skipped, then result is skipped
* any cancelled, then result cancelled
2024-12-16 21:49:53 +08:00
d28a4843b8 Fix incomplete Actions status aggregations (#32859)
Some checks are pending
release-nightly / nightly-binary (push) Waiting to run
release-nightly / nightly-docker-rootful (push) Waiting to run
release-nightly / nightly-docker-rootless (push) Waiting to run
cron-translations / crowdin-pull (push) Has been skipped
fix #32857
2024-12-16 11:18:00 +08:00
33e8e82c4b Enable tenv and testifylint rules (#32852)
Enables tenv and testifylint linters
closes: https://github.com/go-gitea/gitea/issues/32842
2024-12-15 10:41:29 +00:00
7269130d28 Fix missing outputs for jobs with matrix (#32823)
Fix #32795

If a job uses a matrix, multiple `ActionRunJobs` may have the same
`JobID`. We need to merge the outputs of these jobs to make them
available to the jobs that need them.
2024-12-14 10:22:30 +08:00
6370d2fb93 Detect whether action view branch was deleted (#32764)
Fix #32761 

![图片](https://github.com/user-attachments/assets/a5a7eef8-0fea-4242-b199-1b0b73d9bbdb)
2024-12-12 19:28:23 +00:00
98d9a71ffe Trim title before insert/update to database to match the size requirements of database (#32498)
Fix #32489
2024-11-14 07:19:14 +00:00
f888e45432 Harden runner updateTask and updateLog api (#32462)
Per proposal https://github.com/go-gitea/gitea/issues/32461
2024-11-11 04:58:37 +00:00
24b83ff63e Fix milestone deadline and date related problems (#32339)
Use zero instead of 9999-12-31 for deadline
Fix #32291

---------

Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
Co-authored-by: Giteabot <teabot@gitea.io>
2024-11-05 07:46:40 +00:00
9116665e9c Always update expiration time when creating an artifact (#32281)
Fix #32256
2024-10-17 09:05:38 +00:00
df27846628 Show latest run when visit /run/latest (#31808)
Proposal from
https://github.com/go-gitea/gitea/issues/27911#issuecomment-2271982172

When visit latest run path, such as
`/{user}/{repo}/actions/runs/latest`. It renders latest run instead of
index=0 currently.
2024-08-10 08:40:41 +08:00
33cc5837a6 Support compression for Actions logs (#31761)
Support compression for Actions logs to save storage space and
bandwidth. Inspired by
https://github.com/go-gitea/gitea/issues/24256#issuecomment-1521153015

The biggest challenge is that the compression format should support
[seekable](https://github.com/facebook/zstd/blob/dev/contrib/seekable_format/zstd_seekable_compression_format.md).
So when users are viewing a part of the log lines, Gitea doesn't need to
download the whole compressed file and decompress it.

That means gzip cannot help here. And I did research, there aren't too
many choices, like bgzip and xz, but I think zstd is the most popular
one. It has an implementation in Golang with
[zstd](https://github.com/klauspost/compress/tree/master/zstd) and
[zstd-seekable-format-go](https://github.com/SaveTheRbtz/zstd-seekable-format-go),
and what is better is that it has good compatibility: a seekable format
zstd file can be read by a regular zstd reader.

This PR introduces a new package `zstd` to combine and wrap the two
packages, to provide a unified and easy-to-use API.

And a new setting `LOG_COMPRESSION` is added to the config, although I
don't see any reason why not to use compression, I think's it's a good
idea to keep the default with `none` to be consistent with old versions.

`LOG_COMPRESSION` takes effect for only new log files, it adds `.zst` as
an extension to the file name, so Gitea can determine if it needs
decompression according to the file name when reading. Old files will
keep the format since it's not worth converting them, as they will be
cleared after #31735.

<img width="541" alt="image"
src="https://github.com/user-attachments/assets/e9598764-a4e0-4b68-8c2b-f769265183c9">
2024-08-09 10:10:30 +08:00
687c118248 Clear up old Actions logs (#31735)
Part of #24256.

Clear up old action logs to free up storage space.

Users will see a message indicating that the log has been cleared if
they view old tasks.

<img width="1361" alt="image"
src="https://github.com/user-attachments/assets/9f0f3a3a-bc5a-402f-90ca-49282d196c22">

Docs: https://gitea.com/gitea/docs/pulls/40

---------

Co-authored-by: silverwind <me@silverwind.io>
2024-08-02 00:42:08 +00:00
21a73ae642 Use UTC as default timezone when schedule Actions cron tasks (#31742)
Fix #31657.

According to the
[doc](https://docs.github.com/en/actions/writing-workflows/workflow-syntax-for-github-actions#onschedule)
of GitHub Actions, The timezone for cron should be UTC, not the local
timezone. And Gitea Actions doesn't have any reasons to change this, so
I think it's a bug.

However, Gitea Actions has extended the syntax, as it supports
descriptors like `@weekly` and `@every 5m`, and supports specifying the
timezone like `TZ=UTC 0 10 * * *`. So we can make it use UTC only when
the timezone is not specified, to be compatible with GitHub Actions, and
also respect the user's specified.

It does break the feature because the times to run tasks would be
changed, and it may confuse users. So I don't think we should backport
this.

## ⚠️ BREAKING ⚠️

If the server's local time zone is not UTC, a scheduled task would run
at a different time after upgrading Gitea to this version.
2024-08-01 10:02:46 +00:00
a33e74d40d Clarify Actions resources ownership (#31724)
Fix #31707.

Also related to #31715.

Some Actions resources could has different types of ownership. It could
be:

- global: all repos and orgs/users can use it.
- org/user level: only the org/user can use it.
- repo level: only the repo can use it.

There are two ways to distinguish org/user level from repo level:
1. `{owner_id: 1, repo_id: 2}` for repo level, and `{owner_id: 1,
repo_id: 0}` for org level.
2. `{owner_id: 0, repo_id: 2}` for repo level, and `{owner_id: 1,
repo_id: 0}` for org level.

The first way seems more reasonable, but it may not be true. The point
is that although a resource, like a runner, belongs to a repo (it can be
used by the repo), the runner doesn't belong to the repo's org (other
repos in the same org cannot use the runner). So, the second method
makes more sense.

And the first way is not user-friendly to query, we must set the repo id
to zero to avoid wrong results.

So, #31715 should be right. And the most simple way to fix #31707 is
just:

```diff
-	shared.GetRegistrationToken(ctx, ctx.Repo.Repository.OwnerID, ctx.Repo.Repository.ID)
+	shared.GetRegistrationToken(ctx, 0, ctx.Repo.Repository.ID)
```

However, it is quite intuitive to set both owner id and repo id since
the repo belongs to the owner. So I prefer to be compatible with it. If
we get both owner id and repo id not zero when creating or finding, it's
very clear that the caller want one with repo level, but set owner id
accidentally. So it's OK to accept it but fix the owner id to zero.
2024-08-01 09:04:04 +00:00
572fa55fbc Drop IDOrderDesc for listing Actions task and always order by id DESC (#31150)
Close #31066

Just follow what `FindRunOptions` and `FindScheduleOptions` do.
2024-05-31 02:30:02 +00:00
9b2536b78f Update misspell to 0.5.1 and add misspellings.csv (#30573)
Misspell 0.5.0 supports passing a csv file to extend the list of
misspellings, so I added some common ones from the codebase. There is at
least one typo in a API response so we need to decided whether to revert
that and then likely remove the dict entry.
2024-04-27 08:03:49 +00:00
2a3906d755 Improve job commit description (#30579)
Fix https://github.com/go-gitea/gitea/issues/30567

When job is a schedule:

![image](https://github.com/go-gitea/gitea/assets/18380374/b07e9d43-e8b7-4ee2-87b3-a7050c3a8ca5)
When it is a normal one:

![image](https://github.com/go-gitea/gitea/assets/18380374/0d58dab9-74bb-421b-8952-0578cdf21a52)

also add a 'space' behind  `:`

![image](https://github.com/go-gitea/gitea/assets/18380374/4cebece0-bfe6-4ad9-b806-e5c49bb9be43)


![image](https://github.com/go-gitea/gitea/assets/18380374/02da7681-474b-4c0f-9dad-b6558f6cb484)

---------

Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
2024-04-26 02:22:45 +00:00
2f6b1c46a1 Interpolate runs-on with variables when scheduling tasks (#30640)
Follow #29468
1. Interpolate runs-on with variables when scheduling tasks.
2. The `GetVariablesOfRun` function will check if the `Repo` of the run
is nil.

---------

Co-authored-by: Giteabot <teabot@gitea.io>
2024-04-23 20:55:25 +02:00
30dd4beeee Add a db consistency check to remove runners that do not belong to a repository (#30614)
Follow #30406
2024-04-23 03:51:52 +00:00
96d31fe0a8 Avoid user does not exist error when detecting schedule actions when the commit author is an external user (#30357)
![image](https://github.com/go-gitea/gitea/assets/18380374/ddf6ee84-2242-49b9-b066-bd8429ba4d76)

When repo is a mirror, and commit author is an external user, then
`GetUserByEmail` will return error.

reproduce/test:
- mirror Gitea to your instance
- disable action and enable it again, this will trigger
`DetectAndHandleSchedules`

ps: also follow #24706, it only fixed normal runs, not scheduled runs.
2024-04-11 15:11:32 +08:00
b09687f1d1 Refactor more filterslice (#30370) 2024-04-10 04:18:41 +00:00
310e2517e5 Fix ambiguous id when fetch Actions tasks (#30382)
Fix regression of #30331.

```txt
time="2024-04-10T02:23:49Z" level=error msg="failed to fetch task" func="[fetchTask]" file="[poller.go:91]" error="unknown: rpc error: code = Internal desc = pick task: CreateTaskForRunner: Error 1052 (23000): Column 'id' in field list is ambiguous"
```
2024-04-10 02:57:43 +00:00
d547b53cca Add container.FilterSlice function (#30339)
Many places have the following logic:
```go
func (jobs ActionJobList) GetRunIDs() []int64 {
	ids := make(container.Set[int64], len(jobs))
	for _, j := range jobs {
		if j.RunID == 0 {
			continue
		}
		ids.Add(j.RunID)
	}
	return ids.Values()
}
```

this introduces a `container.FilterMapUnique` function, which reduces
the code above to:
```go
func (jobs ActionJobList) GetRunIDs() []int64 {
	return container.FilterMapUnique(jobs, func(j *ActionRunJob) (int64, bool) {
		return j.RunID, j.RunID != 0
	})
}
```
2024-04-09 20:27:30 +08:00
d872ce006c Avoid running action when action unit is disabled after workflows detected (#30331)
Fix #30243

We only checking unit disabled when detecting workflows, but not in
runner `FetchTask`.
So if a workflow was detected when action unit is enabled, but disabled
later, `FetchTask` will still return these detected actions.

Global setting: repo.ENABLED and repository.`DISABLED_REPO_UNITS` will
not effect this.
2024-04-08 22:08:26 +08:00
62b073e6f3 Add API for Variables (#29520)
close #27801

---------

Co-authored-by: silverwind <me@silverwind.io>
2024-03-28 20:40:35 +00:00
b150ff0bab Cancel previous runs of the same PR automatically (#29961)
Follow #25716. Also cancel previous runs for `pull_request_sync`.

It's not a bug since it original PR said "if the event is push".

The main change is
https://github.com/go-gitea/gitea/pull/29961/files#diff-08adda3f8ae0360937f46abb1f4418603bd3518522baa356be11c6c7ac4abcc3.

And also rename `CancelRunningJobs` to `CancelPreviousJobs` to make it
more clear.
2024-03-21 15:01:35 +08:00