You can not select more than 25 topics Topics must start with a chinese character,a letter or number, can include dashes ('-') and can be up to 35 characters long.

migrations.go 23 kB

10 years ago
Oauth2 consumer (#679) * initial stuff for oauth2 login, fails on: * login button on the signIn page to start the OAuth2 flow and a callback for each provider Only GitHub is implemented for now * show login button only when the OAuth2 consumer is configured (and activated) * create macaron group for oauth2 urls * prevent net/http in modules (other then oauth2) * use a new data sessions oauth2 folder for storing the oauth2 session data * add missing 2FA when this is enabled on the user * add password option for OAuth2 user , for use with git over http and login to the GUI * add tip for registering a GitHub OAuth application * at startup of Gitea register all configured providers and also on adding/deleting of new providers * custom handling of errors in oauth2 request init + show better tip * add ExternalLoginUser model and migration script to add it to database * link a external account to an existing account (still need to handle wrong login and signup) and remove if user is removed * remove the linked external account from the user his settings * if user is unknown we allow him to register a new account or link it to some existing account * sign up with button on signin page (als change OAuth2Provider structure so we can store basic stuff about providers) * from gorilla/sessions docs: "Important Note: If you aren't using gorilla/mux, you need to wrap your handlers with context.ClearHandler as or else you will leak memory!" (we're using gorilla/sessions for storing oauth2 sessions) * use updated goth lib that now supports getting the OAuth2 user if the AccessToken is still valid instead of re-authenticating (prevent flooding the OAuth2 provider)
8 years ago
Feature: Timetracking (#2211) * Added comment's hashtag to url for mail notifications. * Added explanation to return statement + documentation. * Replacing in-line link generation with HTMLURL. (+gofmt) * Replaced action-based model with nil-based model. (+gofmt) * Replaced mailIssueActionToParticipants with mailIssueCommentToParticipants. * Updating comment for mailIssueCommentToParticipants * Added link to comment in "Dashboard" * Deleting feed entry if a comment is going to be deleted * Added migration * Added improved migration to add a CommentID column to action. * Added improved links to comments in feed entries. * Fixes #1956 by filtering for deleted comments that are referenced in actions. * Introducing "IsDeleted" column to action. * Adding design draft (not functional) * Adding database models for stopwatches and trackedtimes * See go-gitea/gitea#967 * Adding design draft (not functional) * Adding translations and improving design * Implementing stopwatch (for timetracking) * Make UI functional * Add hints in timeline for time tracking events * Implementing timetracking feature * Adding "Add time manual" option * Improved stopwatch * Created report of total spent time by user * Only showing total time spent if theire is something to show. * Adding license headers. * Improved error handling for "Add Time Manual" * Adding @sapks 's changes, refactoring * Adding API for feature tracking * Adding unit test * Adding DISABLE/ENABLE option to Repository settings page * Improving translations * Applying @sapk 's changes * Removing repo_unit and using IssuesSetting for disabling/enabling timetracker * Adding DEFAULT_ENABLE_TIMETRACKER to config, installation and admin menu * Improving documentation * Fixing vendor/ folder * Changing timtracking routes by adding subgroups /times and /times/stopwatch (Proposed by @lafriks ) * Restricting write access to timetracking based on the repo settings (Proposed by @lafriks ) * Fixed minor permissions bug. * Adding CanUseTimetracker and IsTimetrackerEnabled in ctx.Repo * Allow assignees and authors to track there time too. * Fixed some build-time-errors + logical errors. * Removing unused Get...ByID functions * Moving IsTimetrackerEnabled from context.Repository to models.Repository * Adding a seperate file for issue related repo functions * Adding license headers * Fixed GetUserByParams return 404 * Moving /users/:username/times to /repos/:username/:reponame/times/:username for security reasons * Adding /repos/:username/times to get all tracked times of the repo * Updating sdk-dependency * Updating swagger.v1.json * Adding warning if user has already a running stopwatch (auto-timetracker) * Replacing GetTrackedTimesBy... with GetTrackedTimes(options FindTrackedTimesOptions) * Changing code.gitea.io/sdk back to code.gitea.io/sdk * Correcting spelling mistake * Updating vendor.json * Changing GET stopwatch/toggle to POST stopwatch/toggle * Changing GET stopwatch/cancel to POST stopwatch/cancel * Added migration for stopwatches/timetracking * Fixed some access bugs for read-only users * Added default allow only contributors to track time value to config * Fixed migration by chaging x.Iterate to x.Find * Resorted imports * Moved Add Time Manually form to repo_form.go * Removed "Seconds" field from Add Time Manually * Resorted imports * Improved permission checking * Fixed some bugs * Added integration test * gofmt * Adding integration test by @lafriks * Added created_unix to comment fixtures * Using last event instead of a fixed event * Adding another integration test by @lafriks * Fixing bug Timetracker enabled causing error 500 at sidebar.tpl * Fixed a refactoring bug that resulted in hiding "HasUserStopwatch" warning. * Returning TrackedTime instead of AddTimeOption at AddTime. * Updating SDK from go-gitea/go-sdk#69 * Resetting Go-SDK back to default repository * Fixing test-vendor by changing ini back to original repository * Adding "tags" to swagger spec * govendor sync * Removed duplicate * Formatting templates * Adding IsTimetrackingEnabled checks to API * Improving translations / english texts * Improving documentation * Updating swagger spec * Fixing integration test caused be translation-changes * Removed encoding issues in local_en-US.ini. * "Added" copyright line * Moved unit.IssuesConfig().EnableTimetracker into a != nil check * Removed some other encoding issues in local_en-US.ini * Improved javascript by checking if data-context exists * Replaced manual comment creation with CreateComment * Removed unnecessary code * Improved error checking * Small cosmetic changes * Replaced int>string>duration parsing with int>duration parsing * Fixed encoding issues * Removed unused imports Signed-off-by: Jonas Franz <info@jonasfranz.software>
8 years ago
10 years ago
10 years ago
10 years ago
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899100101102103104105106107108109110111112113114115116117118119120121122123124125126127128129130131132133134135136137138139140141142143144145146147148149150151152153154155156157158159160161162163164165166167168169170171172173174175176177178179180181182183184185186187188189190191192193194195196197198199200201202203204205206207208209210211212213214215216217218219220221222223224225226227228229230231232233234235236237238239240241242243244245246247248249250251252253254255256257258259260261262263264265266267268269270271272273274275276277278279280281282283284285286287288289290291292293294295296297298299300301302303304305306307308309310311312313314315316317318319320321322323324325326327328329330331332333334335336337338339340341342343344345346347348349350351352353354355356357358359360361362363364365366367368369370371372373374375376377378379380381382383384385386387388389390391392393394395396397398399400401402403404405406407408409410411412413414415416417418419420421422423424425426427428429430431432433434435436437438439440441442443444445446447448449450451452453454455456457458459460461462463464465466467468469470471472473474475476477478479480481482483484485486487488489490491492493494495496497498499500501502503504505506507508509510511512513514515516517518519520521522523524525526527528529530531532533534535536537538539540541542543544545546547548549550551552553554555556557558559560561562563564565566567568569570571572573574575576577578579580581582583584585586587588589590591592593594595596597598599600601602603604605606607608609610611612613614615616617618619620621622623624625626627628629630631632633634635636637638639640641642643644645646647648649650651652653654655656657658659660661662663664665666667668669670671672673674675676677678679680681682683684685686687688689690691692693694695696697698699700701702703704705706707708709710711712713714715716717718719720721722723724725726727728729730731732733734735736737738739740741742743744745746747748749750751752753754755756757758759760761762763764765766767768769770771772773774775776777778779780781782783784785786
  1. // Copyright 2015 The Gogs Authors. All rights reserved.
  2. // Use of this source code is governed by a MIT-style
  3. // license that can be found in the LICENSE file.
  4. package migrations
  5. import (
  6. "bytes"
  7. "encoding/json"
  8. "fmt"
  9. "io/ioutil"
  10. "os"
  11. "path"
  12. "path/filepath"
  13. "strings"
  14. "time"
  15. "github.com/Unknwon/com"
  16. "github.com/go-xorm/xorm"
  17. gouuid "github.com/satori/go.uuid"
  18. "gopkg.in/ini.v1"
  19. "code.gitea.io/gitea/modules/base"
  20. "code.gitea.io/gitea/modules/log"
  21. "code.gitea.io/gitea/modules/setting"
  22. )
  23. const minDBVersion = 4
  24. // Migration describes on migration from lower version to high version
  25. type Migration interface {
  26. Description() string
  27. Migrate(*xorm.Engine) error
  28. }
  29. type migration struct {
  30. description string
  31. migrate func(*xorm.Engine) error
  32. }
  33. // NewMigration creates a new migration
  34. func NewMigration(desc string, fn func(*xorm.Engine) error) Migration {
  35. return &migration{desc, fn}
  36. }
  37. // Description returns the migration's description
  38. func (m *migration) Description() string {
  39. return m.description
  40. }
  41. // Migrate executes the migration
  42. func (m *migration) Migrate(x *xorm.Engine) error {
  43. return m.migrate(x)
  44. }
  45. // Version describes the version table. Should have only one row with id==1
  46. type Version struct {
  47. ID int64 `xorm:"pk autoincr"`
  48. Version int64
  49. }
  50. // This is a sequence of migrations. Add new migrations to the bottom of the list.
  51. // If you want to "retire" a migration, remove it from the top of the list and
  52. // update minDBVersion accordingly
  53. var migrations = []Migration{
  54. // v0 -> v4: before 0.6.0 -> 0.7.33
  55. NewMigration("fix locale file load panic", fixLocaleFileLoadPanic), // V4 -> V5:v0.6.0
  56. NewMigration("trim action compare URL prefix", trimCommitActionAppURLPrefix), // V5 -> V6:v0.6.3
  57. NewMigration("generate issue-label from issue", issueToIssueLabel), // V6 -> V7:v0.6.4
  58. NewMigration("refactor attachment table", attachmentRefactor), // V7 -> V8:v0.6.4
  59. NewMigration("rename pull request fields", renamePullRequestFields), // V8 -> V9:v0.6.16
  60. NewMigration("clean up migrate repo info", cleanUpMigrateRepoInfo), // V9 -> V10:v0.6.20
  61. NewMigration("generate rands and salt for organizations", generateOrgRandsAndSalt), // V10 -> V11:v0.8.5
  62. NewMigration("convert date to unix timestamp", convertDateToUnix), // V11 -> V12:v0.9.2
  63. NewMigration("convert LDAP UseSSL option to SecurityProtocol", ldapUseSSLToSecurityProtocol), // V12 -> V13:v0.9.37
  64. // v13 -> v14:v0.9.87
  65. NewMigration("set comment updated with created", setCommentUpdatedWithCreated),
  66. // v14 -> v15
  67. NewMigration("create user column diff view style", createUserColumnDiffViewStyle),
  68. // v15 -> v16
  69. NewMigration("create user column allow create organization", createAllowCreateOrganizationColumn),
  70. // V16 -> v17
  71. NewMigration("create repo unit table and add units for all repos", addUnitsToTables),
  72. // v17 -> v18
  73. NewMigration("set protect branches updated with created", setProtectedBranchUpdatedWithCreated),
  74. // v18 -> v19
  75. NewMigration("add external login user", addExternalLoginUser),
  76. // v19 -> v20
  77. NewMigration("generate and migrate Git hooks", generateAndMigrateGitHooks),
  78. // v20 -> v21
  79. NewMigration("use new avatar path name for security reason", useNewNameAvatars),
  80. // v21 -> v22
  81. NewMigration("rewrite authorized_keys file via new format", useNewPublickeyFormat),
  82. // v22 -> v23
  83. NewMigration("generate and migrate wiki Git hooks", generateAndMigrateWikiGitHooks),
  84. // v23 -> v24
  85. NewMigration("add user openid table", addUserOpenID),
  86. // v24 -> v25
  87. NewMigration("change the key_id and primary_key_id type", changeGPGKeysColumns),
  88. // v25 -> v26
  89. NewMigration("add show field in user openid table", addUserOpenIDShow),
  90. // v26 -> v27
  91. NewMigration("generate and migrate repo and wiki Git hooks", generateAndMigrateGitHookChains),
  92. // v27 -> v28
  93. NewMigration("change mirror interval from hours to time.Duration", convertIntervalToDuration),
  94. // v28 -> v29
  95. NewMigration("add field for repo size", addRepoSize),
  96. // v29 -> v30
  97. NewMigration("add commit status table", addCommitStatus),
  98. // v30 -> 31
  99. NewMigration("add primary key to external login user", addExternalLoginUserPK),
  100. // v31 -> 32
  101. NewMigration("add field for login source synchronization", addLoginSourceSyncEnabledColumn),
  102. // v32 -> v33
  103. NewMigration("add units for team", addUnitsToRepoTeam),
  104. // v33 -> v34
  105. NewMigration("remove columns from action", removeActionColumns),
  106. // v34 -> v35
  107. NewMigration("give all units to owner teams", giveAllUnitsToOwnerTeams),
  108. // v35 -> v36
  109. NewMigration("adds comment to an action", addCommentIDToAction),
  110. // v36 -> v37
  111. NewMigration("regenerate git hooks", regenerateGitHooks36),
  112. // v37 -> v38
  113. NewMigration("unescape user full names", unescapeUserFullNames),
  114. // v38 -> v39
  115. NewMigration("remove commits and settings unit types", removeCommitsUnitType),
  116. // v39 -> v40
  117. NewMigration("adds time tracking and stopwatches", addTimetracking),
  118. // v40 -> v41
  119. NewMigration("migrate protected branch struct", migrateProtectedBranchStruct),
  120. // v41 -> v42
  121. NewMigration("add default value to user prohibit_login", addDefaultValueToUserProhibitLogin),
  122. // v42 -> v43
  123. NewMigration("add tags to releases and sync existing repositories", releaseAddColumnIsTagAndSyncTags),
  124. // v43 -> v44
  125. NewMigration("fix protected branch can push value to false", fixProtectedBranchCanPushValue),
  126. // v44 -> v45
  127. NewMigration("remove duplicate unit types", removeDuplicateUnitTypes),
  128. // v45 -> v46
  129. NewMigration("remove index column from repo_unit table", removeIndexColumnFromRepoUnitTable),
  130. // v46 -> v47
  131. NewMigration("remove organization watch repositories", removeOrganizationWatchRepo),
  132. // v47 -> v48
  133. NewMigration("add deleted branches", addDeletedBranch),
  134. // v48 -> v49
  135. NewMigration("add repo indexer status", addRepoIndexerStatus),
  136. // v49 -> v50
  137. NewMigration("add lfs lock table", addLFSLock),
  138. }
  139. // Migrate database to current version
  140. func Migrate(x *xorm.Engine) error {
  141. if err := x.Sync(new(Version)); err != nil {
  142. return fmt.Errorf("sync: %v", err)
  143. }
  144. currentVersion := &Version{ID: 1}
  145. has, err := x.Get(currentVersion)
  146. if err != nil {
  147. return fmt.Errorf("get: %v", err)
  148. } else if !has {
  149. // If the version record does not exist we think
  150. // it is a fresh installation and we can skip all migrations.
  151. currentVersion.ID = 0
  152. currentVersion.Version = int64(minDBVersion + len(migrations))
  153. if _, err = x.InsertOne(currentVersion); err != nil {
  154. return fmt.Errorf("insert: %v", err)
  155. }
  156. }
  157. v := currentVersion.Version
  158. if minDBVersion > v {
  159. log.Fatal(4, `Gitea no longer supports auto-migration from your previously installed version.
  160. Please try to upgrade to a lower version (>= v0.6.0) first, then upgrade to current version.`)
  161. return nil
  162. }
  163. if int(v-minDBVersion) > len(migrations) {
  164. // User downgraded Gitea.
  165. currentVersion.Version = int64(len(migrations) + minDBVersion)
  166. _, err = x.ID(1).Update(currentVersion)
  167. return err
  168. }
  169. for i, m := range migrations[v-minDBVersion:] {
  170. log.Info("Migration: %s", m.Description())
  171. if err = m.Migrate(x); err != nil {
  172. return fmt.Errorf("do migrate: %v", err)
  173. }
  174. currentVersion.Version = v + int64(i) + 1
  175. if _, err = x.ID(1).Update(currentVersion); err != nil {
  176. return err
  177. }
  178. }
  179. return nil
  180. }
  181. func fixLocaleFileLoadPanic(_ *xorm.Engine) error {
  182. cfg, err := ini.Load(setting.CustomConf)
  183. if err != nil {
  184. return fmt.Errorf("load custom config: %v", err)
  185. }
  186. cfg.DeleteSection("i18n")
  187. if err = cfg.SaveTo(setting.CustomConf); err != nil {
  188. return fmt.Errorf("save custom config: %v", err)
  189. }
  190. setting.Langs = strings.Split(strings.Replace(strings.Join(setting.Langs, ","), "fr-CA", "fr-FR", 1), ",")
  191. return nil
  192. }
  193. func trimCommitActionAppURLPrefix(x *xorm.Engine) error {
  194. type PushCommit struct {
  195. Sha1 string
  196. Message string
  197. AuthorEmail string
  198. AuthorName string
  199. }
  200. type PushCommits struct {
  201. Len int
  202. Commits []*PushCommit
  203. CompareURL string `json:"CompareUrl"`
  204. }
  205. type Action struct {
  206. ID int64 `xorm:"pk autoincr"`
  207. Content string `xorm:"TEXT"`
  208. }
  209. results, err := x.Query("SELECT `id`,`content` FROM `action` WHERE `op_type`=?", 5)
  210. if err != nil {
  211. return fmt.Errorf("select commit actions: %v", err)
  212. }
  213. sess := x.NewSession()
  214. defer sess.Close()
  215. if err = sess.Begin(); err != nil {
  216. return err
  217. }
  218. var pushCommits *PushCommits
  219. for _, action := range results {
  220. actID := com.StrTo(string(action["id"])).MustInt64()
  221. if actID == 0 {
  222. continue
  223. }
  224. pushCommits = new(PushCommits)
  225. if err = json.Unmarshal(action["content"], pushCommits); err != nil {
  226. return fmt.Errorf("unmarshal action content[%d]: %v", actID, err)
  227. }
  228. infos := strings.Split(pushCommits.CompareURL, "/")
  229. if len(infos) <= 4 {
  230. continue
  231. }
  232. pushCommits.CompareURL = strings.Join(infos[len(infos)-4:], "/")
  233. p, err := json.Marshal(pushCommits)
  234. if err != nil {
  235. return fmt.Errorf("marshal action content[%d]: %v", actID, err)
  236. }
  237. if _, err = sess.Id(actID).Update(&Action{
  238. Content: string(p),
  239. }); err != nil {
  240. return fmt.Errorf("update action[%d]: %v", actID, err)
  241. }
  242. }
  243. return sess.Commit()
  244. }
  245. func issueToIssueLabel(x *xorm.Engine) error {
  246. type IssueLabel struct {
  247. ID int64 `xorm:"pk autoincr"`
  248. IssueID int64 `xorm:"UNIQUE(s)"`
  249. LabelID int64 `xorm:"UNIQUE(s)"`
  250. }
  251. issueLabels := make([]*IssueLabel, 0, 50)
  252. results, err := x.Query("SELECT `id`,`label_ids` FROM `issue`")
  253. if err != nil {
  254. if strings.Contains(err.Error(), "no such column") ||
  255. strings.Contains(err.Error(), "Unknown column") {
  256. return nil
  257. }
  258. return fmt.Errorf("select issues: %v", err)
  259. }
  260. for _, issue := range results {
  261. issueID := com.StrTo(issue["id"]).MustInt64()
  262. // Just in case legacy code can have duplicated IDs for same label.
  263. mark := make(map[int64]bool)
  264. for _, idStr := range strings.Split(string(issue["label_ids"]), "|") {
  265. labelID := com.StrTo(strings.TrimPrefix(idStr, "$")).MustInt64()
  266. if labelID == 0 || mark[labelID] {
  267. continue
  268. }
  269. mark[labelID] = true
  270. issueLabels = append(issueLabels, &IssueLabel{
  271. IssueID: issueID,
  272. LabelID: labelID,
  273. })
  274. }
  275. }
  276. sess := x.NewSession()
  277. defer sess.Close()
  278. if err = sess.Begin(); err != nil {
  279. return err
  280. }
  281. if err = sess.Sync2(new(IssueLabel)); err != nil {
  282. return fmt.Errorf("Sync2: %v", err)
  283. } else if _, err = sess.Insert(issueLabels); err != nil {
  284. return fmt.Errorf("insert issue-labels: %v", err)
  285. }
  286. return sess.Commit()
  287. }
  288. func attachmentRefactor(x *xorm.Engine) error {
  289. type Attachment struct {
  290. ID int64 `xorm:"pk autoincr"`
  291. UUID string `xorm:"uuid INDEX"`
  292. // For rename purpose.
  293. Path string `xorm:"-"`
  294. NewPath string `xorm:"-"`
  295. }
  296. results, err := x.Query("SELECT * FROM `attachment`")
  297. if err != nil {
  298. return fmt.Errorf("select attachments: %v", err)
  299. }
  300. attachments := make([]*Attachment, 0, len(results))
  301. for _, attach := range results {
  302. if !com.IsExist(string(attach["path"])) {
  303. // If the attachment is already missing, there is no point to update it.
  304. continue
  305. }
  306. attachments = append(attachments, &Attachment{
  307. ID: com.StrTo(attach["id"]).MustInt64(),
  308. UUID: gouuid.NewV4().String(),
  309. Path: string(attach["path"]),
  310. })
  311. }
  312. sess := x.NewSession()
  313. defer sess.Close()
  314. if err = sess.Begin(); err != nil {
  315. return err
  316. }
  317. if err = sess.Sync2(new(Attachment)); err != nil {
  318. return fmt.Errorf("Sync2: %v", err)
  319. }
  320. // Note: Roll back for rename can be a dead loop,
  321. // so produces a backup file.
  322. var buf bytes.Buffer
  323. buf.WriteString("# old path -> new path\n")
  324. // Update database first because this is where error happens the most often.
  325. for _, attach := range attachments {
  326. if _, err = sess.Id(attach.ID).Update(attach); err != nil {
  327. return err
  328. }
  329. attach.NewPath = path.Join(setting.AttachmentPath, attach.UUID[0:1], attach.UUID[1:2], attach.UUID)
  330. buf.WriteString(attach.Path)
  331. buf.WriteString("\t")
  332. buf.WriteString(attach.NewPath)
  333. buf.WriteString("\n")
  334. }
  335. // Then rename attachments.
  336. isSucceed := true
  337. defer func() {
  338. if isSucceed {
  339. return
  340. }
  341. dumpPath := path.Join(setting.LogRootPath, "attachment_path.dump")
  342. ioutil.WriteFile(dumpPath, buf.Bytes(), 0666)
  343. log.Info("Failed to rename some attachments, old and new paths are saved into: %s", dumpPath)
  344. }()
  345. for _, attach := range attachments {
  346. if err = os.MkdirAll(path.Dir(attach.NewPath), os.ModePerm); err != nil {
  347. isSucceed = false
  348. return err
  349. }
  350. if err = os.Rename(attach.Path, attach.NewPath); err != nil {
  351. isSucceed = false
  352. return err
  353. }
  354. }
  355. return sess.Commit()
  356. }
  357. func renamePullRequestFields(x *xorm.Engine) (err error) {
  358. type PullRequest struct {
  359. ID int64 `xorm:"pk autoincr"`
  360. PullID int64 `xorm:"INDEX"`
  361. PullIndex int64
  362. HeadBarcnh string
  363. IssueID int64 `xorm:"INDEX"`
  364. Index int64
  365. HeadBranch string
  366. }
  367. if err = x.Sync(new(PullRequest)); err != nil {
  368. return fmt.Errorf("sync: %v", err)
  369. }
  370. results, err := x.Query("SELECT `id`,`pull_id`,`pull_index`,`head_barcnh` FROM `pull_request`")
  371. if err != nil {
  372. if strings.Contains(err.Error(), "no such column") {
  373. return nil
  374. }
  375. return fmt.Errorf("select pull requests: %v", err)
  376. }
  377. sess := x.NewSession()
  378. defer sess.Close()
  379. if err = sess.Begin(); err != nil {
  380. return err
  381. }
  382. var pull *PullRequest
  383. for _, pr := range results {
  384. pull = &PullRequest{
  385. ID: com.StrTo(pr["id"]).MustInt64(),
  386. IssueID: com.StrTo(pr["pull_id"]).MustInt64(),
  387. Index: com.StrTo(pr["pull_index"]).MustInt64(),
  388. HeadBranch: string(pr["head_barcnh"]),
  389. }
  390. if pull.Index == 0 {
  391. continue
  392. }
  393. if _, err = sess.Id(pull.ID).Update(pull); err != nil {
  394. return err
  395. }
  396. }
  397. return sess.Commit()
  398. }
  399. func cleanUpMigrateRepoInfo(x *xorm.Engine) (err error) {
  400. type (
  401. User struct {
  402. ID int64 `xorm:"pk autoincr"`
  403. LowerName string
  404. }
  405. Repository struct {
  406. ID int64 `xorm:"pk autoincr"`
  407. OwnerID int64
  408. LowerName string
  409. }
  410. )
  411. repos := make([]*Repository, 0, 25)
  412. if err = x.Where("is_mirror=?", false).Find(&repos); err != nil {
  413. return fmt.Errorf("select all non-mirror repositories: %v", err)
  414. }
  415. var user *User
  416. for _, repo := range repos {
  417. user = &User{ID: repo.OwnerID}
  418. has, err := x.Get(user)
  419. if err != nil {
  420. return fmt.Errorf("get owner of repository[%d - %d]: %v", repo.ID, repo.OwnerID, err)
  421. } else if !has {
  422. continue
  423. }
  424. configPath := filepath.Join(setting.RepoRootPath, user.LowerName, repo.LowerName+".git/config")
  425. // In case repository file is somehow missing.
  426. if !com.IsFile(configPath) {
  427. continue
  428. }
  429. cfg, err := ini.Load(configPath)
  430. if err != nil {
  431. return fmt.Errorf("open config file: %v", err)
  432. }
  433. cfg.DeleteSection("remote \"origin\"")
  434. if err = cfg.SaveToIndent(configPath, "\t"); err != nil {
  435. return fmt.Errorf("save config file: %v", err)
  436. }
  437. }
  438. return nil
  439. }
  440. func generateOrgRandsAndSalt(x *xorm.Engine) (err error) {
  441. type User struct {
  442. ID int64 `xorm:"pk autoincr"`
  443. Rands string `xorm:"VARCHAR(10)"`
  444. Salt string `xorm:"VARCHAR(10)"`
  445. }
  446. orgs := make([]*User, 0, 10)
  447. if err = x.Where("type=1").And("rands=''").Find(&orgs); err != nil {
  448. return fmt.Errorf("select all organizations: %v", err)
  449. }
  450. sess := x.NewSession()
  451. defer sess.Close()
  452. if err = sess.Begin(); err != nil {
  453. return err
  454. }
  455. for _, org := range orgs {
  456. if org.Rands, err = base.GetRandomString(10); err != nil {
  457. return err
  458. }
  459. if org.Salt, err = base.GetRandomString(10); err != nil {
  460. return err
  461. }
  462. if _, err = sess.Id(org.ID).Update(org); err != nil {
  463. return err
  464. }
  465. }
  466. return sess.Commit()
  467. }
  468. // TAction defines the struct for migrating table action
  469. type TAction struct {
  470. ID int64 `xorm:"pk autoincr"`
  471. CreatedUnix int64
  472. }
  473. // TableName will be invoked by XORM to customrize the table name
  474. func (t *TAction) TableName() string { return "action" }
  475. // TNotice defines the struct for migrating table notice
  476. type TNotice struct {
  477. ID int64 `xorm:"pk autoincr"`
  478. CreatedUnix int64
  479. }
  480. // TableName will be invoked by XORM to customrize the table name
  481. func (t *TNotice) TableName() string { return "notice" }
  482. // TComment defines the struct for migrating table comment
  483. type TComment struct {
  484. ID int64 `xorm:"pk autoincr"`
  485. CreatedUnix int64
  486. }
  487. // TableName will be invoked by XORM to customrize the table name
  488. func (t *TComment) TableName() string { return "comment" }
  489. // TIssue defines the struct for migrating table issue
  490. type TIssue struct {
  491. ID int64 `xorm:"pk autoincr"`
  492. DeadlineUnix int64
  493. CreatedUnix int64
  494. UpdatedUnix int64
  495. }
  496. // TableName will be invoked by XORM to customrize the table name
  497. func (t *TIssue) TableName() string { return "issue" }
  498. // TMilestone defines the struct for migrating table milestone
  499. type TMilestone struct {
  500. ID int64 `xorm:"pk autoincr"`
  501. DeadlineUnix int64
  502. ClosedDateUnix int64
  503. }
  504. // TableName will be invoked by XORM to customrize the table name
  505. func (t *TMilestone) TableName() string { return "milestone" }
  506. // TAttachment defines the struct for migrating table attachment
  507. type TAttachment struct {
  508. ID int64 `xorm:"pk autoincr"`
  509. CreatedUnix int64
  510. }
  511. // TableName will be invoked by XORM to customrize the table name
  512. func (t *TAttachment) TableName() string { return "attachment" }
  513. // TLoginSource defines the struct for migrating table login_source
  514. type TLoginSource struct {
  515. ID int64 `xorm:"pk autoincr"`
  516. CreatedUnix int64
  517. UpdatedUnix int64
  518. }
  519. // TableName will be invoked by XORM to customrize the table name
  520. func (t *TLoginSource) TableName() string { return "login_source" }
  521. // TPull defines the struct for migrating table pull_request
  522. type TPull struct {
  523. ID int64 `xorm:"pk autoincr"`
  524. MergedUnix int64
  525. }
  526. // TableName will be invoked by XORM to customrize the table name
  527. func (t *TPull) TableName() string { return "pull_request" }
  528. // TRelease defines the struct for migrating table release
  529. type TRelease struct {
  530. ID int64 `xorm:"pk autoincr"`
  531. CreatedUnix int64
  532. }
  533. // TableName will be invoked by XORM to customrize the table name
  534. func (t *TRelease) TableName() string { return "release" }
  535. // TRepo defines the struct for migrating table repository
  536. type TRepo struct {
  537. ID int64 `xorm:"pk autoincr"`
  538. CreatedUnix int64
  539. UpdatedUnix int64
  540. }
  541. // TableName will be invoked by XORM to customrize the table name
  542. func (t *TRepo) TableName() string { return "repository" }
  543. // TMirror defines the struct for migrating table mirror
  544. type TMirror struct {
  545. ID int64 `xorm:"pk autoincr"`
  546. UpdatedUnix int64
  547. NextUpdateUnix int64
  548. }
  549. // TableName will be invoked by XORM to customrize the table name
  550. func (t *TMirror) TableName() string { return "mirror" }
  551. // TPublicKey defines the struct for migrating table public_key
  552. type TPublicKey struct {
  553. ID int64 `xorm:"pk autoincr"`
  554. CreatedUnix int64
  555. UpdatedUnix int64
  556. }
  557. // TableName will be invoked by XORM to customrize the table name
  558. func (t *TPublicKey) TableName() string { return "public_key" }
  559. // TDeployKey defines the struct for migrating table deploy_key
  560. type TDeployKey struct {
  561. ID int64 `xorm:"pk autoincr"`
  562. CreatedUnix int64
  563. UpdatedUnix int64
  564. }
  565. // TableName will be invoked by XORM to customrize the table name
  566. func (t *TDeployKey) TableName() string { return "deploy_key" }
  567. // TAccessToken defines the struct for migrating table access_token
  568. type TAccessToken struct {
  569. ID int64 `xorm:"pk autoincr"`
  570. CreatedUnix int64
  571. UpdatedUnix int64
  572. }
  573. // TableName will be invoked by XORM to customrize the table name
  574. func (t *TAccessToken) TableName() string { return "access_token" }
  575. // TUser defines the struct for migrating table user
  576. type TUser struct {
  577. ID int64 `xorm:"pk autoincr"`
  578. CreatedUnix int64
  579. UpdatedUnix int64
  580. }
  581. // TableName will be invoked by XORM to customrize the table name
  582. func (t *TUser) TableName() string { return "user" }
  583. // TWebhook defines the struct for migrating table webhook
  584. type TWebhook struct {
  585. ID int64 `xorm:"pk autoincr"`
  586. CreatedUnix int64
  587. UpdatedUnix int64
  588. }
  589. // TableName will be invoked by XORM to customrize the table name
  590. func (t *TWebhook) TableName() string { return "webhook" }
  591. func convertDateToUnix(x *xorm.Engine) (err error) {
  592. log.Info("This migration could take up to minutes, please be patient.")
  593. type Bean struct {
  594. ID int64 `xorm:"pk autoincr"`
  595. Created time.Time
  596. Updated time.Time
  597. Merged time.Time
  598. Deadline time.Time
  599. ClosedDate time.Time
  600. NextUpdate time.Time
  601. }
  602. var tables = []struct {
  603. name string
  604. cols []string
  605. bean interface{}
  606. }{
  607. {"action", []string{"created"}, new(TAction)},
  608. {"notice", []string{"created"}, new(TNotice)},
  609. {"comment", []string{"created"}, new(TComment)},
  610. {"issue", []string{"deadline", "created", "updated"}, new(TIssue)},
  611. {"milestone", []string{"deadline", "closed_date"}, new(TMilestone)},
  612. {"attachment", []string{"created"}, new(TAttachment)},
  613. {"login_source", []string{"created", "updated"}, new(TLoginSource)},
  614. {"pull_request", []string{"merged"}, new(TPull)},
  615. {"release", []string{"created"}, new(TRelease)},
  616. {"repository", []string{"created", "updated"}, new(TRepo)},
  617. {"mirror", []string{"updated", "next_update"}, new(TMirror)},
  618. {"public_key", []string{"created", "updated"}, new(TPublicKey)},
  619. {"deploy_key", []string{"created", "updated"}, new(TDeployKey)},
  620. {"access_token", []string{"created", "updated"}, new(TAccessToken)},
  621. {"user", []string{"created", "updated"}, new(TUser)},
  622. {"webhook", []string{"created", "updated"}, new(TWebhook)},
  623. }
  624. for _, table := range tables {
  625. log.Info("Converting table: %s", table.name)
  626. if err = x.Sync2(table.bean); err != nil {
  627. return fmt.Errorf("Sync [table: %s]: %v", table.name, err)
  628. }
  629. offset := 0
  630. for {
  631. beans := make([]*Bean, 0, 100)
  632. if err = x.Table(table.name).Asc("id").Limit(100, offset).Find(&beans); err != nil {
  633. return fmt.Errorf("select beans [table: %s, offset: %d]: %v", table.name, offset, err)
  634. }
  635. log.Trace("Table [%s]: offset: %d, beans: %d", table.name, offset, len(beans))
  636. if len(beans) == 0 {
  637. break
  638. }
  639. offset += 100
  640. baseSQL := "UPDATE `" + table.name + "` SET "
  641. for _, bean := range beans {
  642. valSQLs := make([]string, 0, len(table.cols))
  643. for _, col := range table.cols {
  644. fieldSQL := ""
  645. fieldSQL += col + "_unix = "
  646. switch col {
  647. case "deadline":
  648. if bean.Deadline.IsZero() {
  649. continue
  650. }
  651. fieldSQL += com.ToStr(bean.Deadline.Unix())
  652. case "created":
  653. fieldSQL += com.ToStr(bean.Created.Unix())
  654. case "updated":
  655. fieldSQL += com.ToStr(bean.Updated.Unix())
  656. case "closed_date":
  657. fieldSQL += com.ToStr(bean.ClosedDate.Unix())
  658. case "merged":
  659. fieldSQL += com.ToStr(bean.Merged.Unix())
  660. case "next_update":
  661. fieldSQL += com.ToStr(bean.NextUpdate.Unix())
  662. }
  663. valSQLs = append(valSQLs, fieldSQL)
  664. }
  665. if len(valSQLs) == 0 {
  666. continue
  667. }
  668. if _, err = x.Exec(baseSQL + strings.Join(valSQLs, ",") + " WHERE id = " + com.ToStr(bean.ID)); err != nil {
  669. return fmt.Errorf("update bean [table: %s, id: %d]: %v", table.name, bean.ID, err)
  670. }
  671. }
  672. }
  673. }
  674. return nil
  675. }