web/notify.moe
Code Issues Pull Requests Activity

Merge pull request #4 from animenotifier/go

Browse Source 
test
This commit is contained in:
Allen Lydiard 2017-11-05 10:32:43 -04:00 committed by GitHub
commit db9d2f5191
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
351 changed files with 11622 additions and 2464 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

1
.gitignore vendored
View File

@ -22,6 +22,7 @@ _testmain.go
*.exe
*.test
*.prof
*.pprof
# Output of the go coverage tool, specifically when used with LiteIDE
*.out

46
CODE_OF_CONDUCT.md Normal file
View File

@ -0,0 +1,46 @@
# Contributor Covenant Code of Conduct
## Our Pledge
In the interest of fostering an open and welcoming environment, we as contributors and maintainers pledge to making participation in our project and our community a harassment-free experience for everyone, regardless of age, body size, disability, ethnicity, gender identity and expression, level of experience, nationality, personal appearance, race, religion, or sexual identity and orientation.
## Our Standards
Examples of behavior that contributes to creating a positive environment include:
* Using welcoming and inclusive language
* Being respectful of differing viewpoints and experiences
* Gracefully accepting constructive criticism
* Focusing on what is best for the community
* Showing empathy towards other community members
Examples of unacceptable behavior by participants include:
* The use of sexualized language or imagery and unwelcome sexual attention or advances
* Trolling, insulting/derogatory comments, and personal or political attacks
* Public or private harassment
* Publishing others' private information, such as a physical or electronic address, without explicit permission
* Other conduct which could reasonably be considered inappropriate in a professional setting
## Our Responsibilities
Project maintainers are responsible for clarifying the standards of acceptable behavior and are expected to take appropriate and fair corrective action in response to any instances of unacceptable behavior.
Project maintainers have the right and responsibility to remove, edit, or reject comments, commits, code, wiki edits, issues, and other contributions that are not aligned to this Code of Conduct, or to ban temporarily or permanently any contributor for other behaviors that they deem inappropriate, threatening, offensive, or harmful.
## Scope
This Code of Conduct applies both within project spaces and in public spaces when an individual is representing the project or its community. Examples of representing a project or community include using an official project e-mail address, posting via an official social media account, or acting as an appointed representative at an online or offline event. Representation of a project may be further defined and clarified by project maintainers.
## Enforcement
Instances of abusive, harassing, or otherwise unacceptable behavior may be reported by contacting the project team at e.urbach@gmail.com. The project team will review and investigate all complaints, and will respond in a way that it deems appropriate to the circumstances. The project team is obligated to maintain confidentiality with regard to the reporter of an incident. Further details of specific enforcement policies may be posted separately.
Project maintainers who do not follow or enforce the Code of Conduct in good faith may face temporary or permanent repercussions as determined by other members of the project's leadership.
## Attribution
This Code of Conduct is adapted from the [Contributor Covenant][homepage], version 1.4, available at [http://contributor-covenant.org/version/1/4][version]
[homepage]: http://contributor-covenant.org
[version]: http://contributor-covenant.org/version/1/4/

7
CONTRIBUTING.md Normal file
View File

@ -0,0 +1,7 @@
# Contributing
Please get in contact with the team on the [Anime Notifier Discord](https://discord.gg/0kimAmMCeXGXuzNF).
We're willing to help with installations and how to get started with contributions.
There are no stupid questions so feel free to ask anything if you encounter any troubles.
If you'd like to install this project locally, take a look at the [Installation](INSTALLATION.md) guide.

50
INSTALLATION.md Normal file
View File

@ -0,0 +1,50 @@
# Installation
## Prerequisites
* Install [Ubuntu](https://www.ubuntu.com/) or any of its derivates
* Install [Go](https://golang.org/dl/) (1.9 or higher)
* Install [TypeScript](https://www.typescriptlang.org/) (2.5 or higher)
## Download the repository and its dependencies
* `go get github.com/animenotifier/notify.moe`
## Build all
* Navigate to the project directory `notify.moe`
* Run `make tools` to install [pack](https://github.com/aerogo/pack) & [run](https://github.com/aerogo/run)
* Run `make ports` to set up local port forwarding *(80 to 4000, 443 to 4001)*
* Run `make all`
## Hosts
* Add `127.0.0.1 beta.notify.moe` to `/etc/hosts`
## HTTPS
* [Create the certificate](https://stackoverflow.com/questions/10175812/how-to-create-a-self-signed-certificate-with-openssl) `notify.moe/security/fullchain.pem` (domain: `beta.notify.moe`)
* Create the private key `notify.moe/security/privkey.pem`
## Browser
* Start Chrome via `google-chrome --ignore-certificate-errors`
## API keys
* Get a Google OAuth 2.0 client key & secret from [console.developers.google.com](https://console.developers.google.com)
* Create the file `notify.moe/security/api-keys.json`:
```json
{
"google": {
"id": "YOUR_KEY",
"secret": "YOUR_SECRET"
}
}
```
## Run
* Start the web server in notify.moe directory: `run`
* Open `https://beta.notify.moe` which should now resolve to localhost

21
LICENSE Normal file
View File

@ -0,0 +1,21 @@
MIT License
Copyright (c) 2017 Eduard Urbach
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.

255
README.md
View File

@ -1,88 +1,225 @@
# Anime Notifier
## Info
## What kind of website is this?
notify.moe is powered by the [Aero framework](https://github.com/aerogo/aero) from the same author. The project also uses Go and Aerospike.
An anime tracker where you can add anime to your list and edit your episode progress using either the website, the chrome extension or the mobile app.
## Installation
## Why is it called notify.moe?
### Prerequisites
Because we made a notifier that takes your watching list, checks it against external websites and notifies you when there is a new episode on that external site. It's also a terrible wordplay combining "notify me!" and [moe](https://en.wikipedia.org/wiki/Moe_(slang)).
* Install a Debian based operating system
* Install [Go](https://golang.org/dl/) (1.9 or higher)
* Install [Aerospike](http://www.aerospike.com/download) (3.14.0 or higher)
## So it's just a notifier?
### Download the repository and its dependencies
In the past it was, but not anymore. We're growing bigger by establishing a database that combines information from multiple sources and also growing as a community. Many of us are hanging out on Discord and you are welcome to join us. We also have our own anime lists now due to popular request of adding episode progress changes to our browser extension.
* `go get github.com/animenotifier/notify.moe`
## What does the current feature set look like?
### Install pack & run
* [Chrome extension](https://chrome.google.com/webstore/detail/anime-notifier/hajchfikckiofgilinkpifobdbiajfch) for quick watching list access and episode updates
* Edit episode progress and rating by clicking on the number
* Airing dates
* Offline browsing
* Push notifications
* Soundtracks
* Anime & user search
* Anime rating system
* [twist.moe](https://twist.moe) integration
* [anilist.co](https://anilist.co/), [myanimelist.net](https://myanimelist.net/) and [kitsu.io](https://kitsu.io/) import
* [osu](https://osu.ppy.sh/) ranking view
* [Gravatar](https://gravatar.com) support
* User profiles
* Dashboard
* Forums
* Responsive layout (looks good on 1080p and on mobile devices)
* `go get github.com/aerogo/pack`
* `go get github.com/aerogo/run`
* `go install github.com/aerogo/pack`
* `go install github.com/aerogo/run`
## Can I follow the project on social media?
### Build all
* [Facebook](https://www.facebook.com/animenotifier)
* [Twitter](https://twitter.com/animenotifier)
* [Google+](https://plus.google.com/+AnimeReleaseNotifierOfficial)
* [GitHub](https://github.com/animenotifier/notify.moe)
* [Discord](https://discord.gg/0kimAmMCeXGXuzNF)
* Run `make all`
* Run `make ports` to set up local port forwarding *(80 to 4000, 443 to 4001)*
* You should be able to start the server by executing `run` now
## How do I enable notifications?
### Database
Use a browser that supports push notifications (Chrome or Firefox). Then go to your [settings](https://notify.moe/settings) and click "Enable notifications". This might take a while, especially on mobile devices. After that you can press "Send test notification". If you get a notification saying "Yay, it works!" then everything's fine. The real thing looks like this:
* Remove all namespaces in `/etc/aerospike/aerospike.conf`
* Add a namespace called `arn`:
![Anime Notifications](https://puu.sh/wKpcm/304a4441a0.png)
```
namespace arn {
storage-engine device {
file /home/YOUR_NAME/YOUR_PATH/notify.moe/db/arn-dev.dat
filesize 64M
data-in-memory true
## How do I use the search?
# Maximum object size. 128K is ideal for SSDs but we need 1M for search indices.
write-block-size 1M
Press the "F" key and start searching for an anime title.
# Write block size x Post write queue = Cache memory usage (for write block buffers)
post-write-queue 1
}
}
```
![Anime search](https://puu.sh/wM45s/ffe5025c63.png)
* Download the [database for developers](https://mega.nz/#!iN4WTRxb!R_cRjBbnUUvGeXdtRGiqbZRrnvy0CHc2MjlyiGBxdP4) to notify.moe/db/arn-dev.dat
* Start the database using `sudo service aerospike start`
* Confirm that the status is "green": `sudo service aerospike status`
## How do I add anime to my list?
### Hosts
Once you open the anime page you should see a button called "Add to my collection". Clicking that will add the anime to your "Plan to watch" list. To move it to your current "Watching" list, you need to click "Edit in collection" and change the status to "Watching".
* Add `127.0.0.1 arn-db` to `/etc/hosts`
* Add `127.0.0.1 beta.notify.moe` to `/etc/hosts`
## How do I edit my episode progress?
### HTTPS
There are 2 ways of editing your progress:
* Create the certificate `notify.moe/security/fullchain.pem` (domain: `beta.notify.moe`)
* Create the private key `notify.moe/security/privkey.pem`
1. Click on the "+" button that shows up when you hover over the episode number. This will increase your progress by one episode on each click.
1. Click on the episode number so that a text input cursor shows up. Use backspace/delete keys and enter your new number directly. Press Enter or click somewhere else to confirm.
### API keys
## How do I edit my rating?
* Get a Google OAuth 2.0 client key & secret from [console.developers.google.com](https://console.developers.google.com)
* Create the file `notify.moe/security/api-keys.json`:
Your "Overall" rating can be edited with the same method as episodes by clicking on the number directly so that the text input cursor shows up, then entering a new number and confirming with Enter. The other 3 rating numbers for Story, Visuals and Soundtrack can only be edited by going into edit mode (click on the anime title in your list).
```json
{
"google": {
"id": "YOUR_KEY",
"secret": "YOUR_SECRET"
}
}
```
## How does the rating system work?
### Fetch data
You can rate each entry in your anime list in 4 different categories:
* Run `jobs/sync-anime/sync-anime` from this repository to fetch anime data
* Overall (this will determine the sorting order)
* Story (how interesting was the story/plot?)
* Visuals (art & effect & animation quality)
* Soundtrack (music rating)
### Run
Each rating is a number on a scale of 0 to 10. A rating of 0 counts as "not rated" and will be ignored in average rating calculations for that anime. Thus the lowest possible rating you can assign to an anime is 0.1. The highest possible rating is 10. The average is close to the number 5.
* Start the web server in notify.moe directory: `run`
* Open `https://beta.notify.moe` which should now resolve to localhost
## What does the Chrome extension offer me?
A quick access to your watching list:
![Anime Notifier Chrome extension](https://puu.sh/wM47V/af25b23755.png)
## How can I format text and include images in the forum?
You need to use [Markdown](https://github.com/adam-p/markdown-here/wiki/Markdown-Cheatsheet).
## What is offline mode?
This website / app is accessible even when you go offline. You can keep browsing the pages you visited earlier which is especially useful for mobile phones or when you're traveling with an unstable connection. Feel free to try it by disabling your WiFi and opening the site while offline.
## Do I need to keep the site open to receive notifications?
No, you can close the site and still receive notifications after you enabled them.
## What are the community rules for conversations on the forum?
* Be respectful to each other.
* Realize that every person has his or her own opinion and that you should treat that opinion with respect. You do not have to agree with strangers on the internet but it's worth thinking about their viewpoint.
* Do not spam.
* Do not advertise unrelated products. If anything it needs to be related to anime or the site itself.
* We do not mind links to competitors or similar websites. Feel free to post them.
## How do I import my former anime list?
We added importers for what we consider to be the 3 most popular list providers:
* anilist.co
* kitsu.io
* myanimelist.net
To use an importer, enter your nickname for the site you want to import from and click the "Import" button with the list provider name that just appeared.
![Anime list import](https://puu.sh/wM4dP/11d43e5f71.png)
## What does following a person do?
You will be able to see their progress and ratings on anime pages:
![Anime pages friends](https://puu.sh/wPfE2/d65ef4f771.png)
## How do I install the site as an Android app?
This website uses a modern technology that allows you to install websites as local apps. To install notify.moe as a mobile app, do the following:
1. Go to https://notify.moe on your Android device.
2. Open the menu by tapping the top right part of your browser.
3. Choose "Add to Home screen" and confirm it.
4. Now you can access your anime list easily from your home screen and get notified about new episodes.
You need to enable notifications on each device separately. To receive notifications on both desktop and mobile phone you need to click "Enable notifications" on both.
## How do I install the site as a PC/desktop app?
In Chrome, open the top right menu and go to **More tools > Add to desktop**. Make sure that "Open as window" is checked.
![Anime Notifier desktop app](https://puu.sh/wM4pB/542add3113.png)
## What do I get notified about?
At the time of writing this, you get notified when:
* A new episode from your watching list is released on twist.moe
* Somebody replies in a thread you have participated in
* Somebody likes your post
* You get a new follower
## How do notifications work from a technical perspective?
There are many, many ways how notifications can be implemented from a technical standpoint. There is e.g. "polling" which means that an app periodically checks external sites and tells you when something new is available. We are not using polling because these periodic checks can quickly drain your battery on a mobile phone. We are using so-called "push notifications" instead. The advantage of push notifications is that your mobile phone or desktop PC doesn't have to do periodic checks anymore - instead the website will send new episode releases to all of your registered devices. This consumes less CPU/network resources and is much more battery friendly for mobile devices.
## How can I confirm I'm a PRO user now?
Go to your [settings](https://notify.moe/settings), it should show you the remaining duration for your [PRO](https://notify.moe/shop) account.
## Is this website well-optimized?
![Anime Notifier - Lighthouse](https://pbs.twimg.com/media/DEplUsNXgAEF-UT.jpg:large)
![Anime Notifier - PageSpeed](https://pbs.twimg.com/media/DEplXmpWsAAPzb6.jpg:large)
## Is this website secure?
* The site is not storing passwords which means there is no password that could be stolen
* The site uses HTTPS, CSP and CSS hashing to improve overall security
* The site functionality is 99.9% server-sided which is a requirement for any security related app
* The site is using only the most modern and secure SSL ciphers
## Is this website mobile-friendly?
Yes, we have a dynamic layout that works on everything from 320p to full HD (1080p). Larger sizes should work well due to automatic layout. On smartphones you can use the sidebar by sliding with your finger to the right side.
## Which platforms and browsers do you officially support?
OS:
* Windows
* Linux
* Mac
Browsers:
* Chrome
* Firefox
* Safari
The most modern browser is [without question](https://html5test.com/compare/browser/chrome-58/firefox-53/safari-10.2.html) Chrome and I highly recommend everyone to switch to Chrome if you're not using it already. Chrome has WebP support which *drastically* reduces page loading times and also lazy loading support which loads images only when they appear in your current viewport, reducing both your bandwidth and your initial loading times.
Firefox and Safari are supported but I do not recommend using them. See these for more information:
* [WebP support](http://caniuse.com/#feat=webp)
* [Push notifications](http://caniuse.com/#feat=push-api)
* [Intersection Observer support](http://caniuse.com/#feat=intersectionobserver) (lazy loading)
* [RequestIdleCallback](http://caniuse.com/#feat=requestidlecallback) (defer unimportant requests to idle times)
## Can you tell me more about the history of this software?
From a technological standpoint we went through quite a few different approaches:
* Version 1.0: This version was just a browser extension with **client-side JS**.
* Version 2.0: To decrease the number of requests/pressure on external sites we made a central website. It was written in **PHP**.
* Version 3.0: A complete remake of the website in **node.js** supporting 4 different list providers and 2 anime providers. Episode changes were not possible.
* Version 4.0: We switched to our own hosted anime lists to make episode updates in the extension as smooth as possible. The website is now written in **Go** and uses 3 separate servers/machines (web server, database and the scheduler).
## How many developers are working on this?
Since 2014 it's been just me, though I do plan to start a company and hire talented people to help me out with this project once the stars align.
## Is there an API for this site?
Yes, the [API](https://notify.moe/api) is an on-going effort and subject to change.
## Can I show my support for this site? Do you accept donations?
I recently added [PRO](https://notify.moe/shop) accounts for an extended feature set. You do not have to donate without getting something back, instead I'd rather be happy to see you profit from the donation as well. It would be my dream to work on this full-time.
## Can I help with coding or change stuff as this is Open Source?
Sure, the setup to start contributing is not that hard. Try to get in contact with me on Discord.
## Can I apply to be a data mod / editor?
Sure, just contact me on Discord if you want to help out with the database.

68
assets.go
View File

@ -1,27 +1,36 @@
package main
import (
"errors"
"net/http"
"io/ioutil"
"strings"
"github.com/aerogo/aero"
"github.com/animenotifier/arn"
"github.com/animenotifier/notify.moe/components/js"
)
func init() {
// Scripts
scripts := js.Bundle()
// configureAssets adds all the routes used for media assets.
func configureAssets(app *aero.Application) {
// Script bundle
scriptBundle := js.Bundle()
// Service worker
serviceWorkerBytes, err := ioutil.ReadFile("sw/service-worker.js")
serviceWorker := string(serviceWorkerBytes)
if err != nil {
panic("Couldn't load service worker")
}
app.Get("/scripts", func(ctx *aero.Context) string {
ctx.SetResponseHeader("Content-Type", "application/javascript")
return scripts
return ctx.JavaScript(scriptBundle)
})
app.Get("/scripts.js", func(ctx *aero.Context) string {
ctx.SetResponseHeader("Content-Type", "application/javascript")
return scripts
return ctx.JavaScript(scriptBundle)
})
app.Get("/service-worker", func(ctx *aero.Context) string {
return ctx.JavaScript(serviceWorker)
})
// Web manifest
@ -36,8 +45,7 @@ func init() {
// Brand icons
app.Get("/images/brand/:file", func(ctx *aero.Context) string {
file := strings.TrimSuffix(ctx.Get("file"), ".webp")
return ctx.TryWebP("images/brand/"+file, ".png")
return ctx.File("images/brand/" + ctx.Get("file"))
})
// Cover image
@ -53,44 +61,12 @@ func init() {
// Avatars
app.Get("/images/avatars/large/:file", func(ctx *aero.Context) string {
file := strings.TrimSuffix(ctx.Get("file"), ".webp")
if ctx.CanUseWebP() {
return ctx.File("images/avatars/large/webp/" + file + ".webp")
}
original := arn.FindFileWithExtension(
file,
"images/avatars/large/original/",
arn.OriginalImageExtensions,
)
if original == "" {
return ctx.Error(http.StatusNotFound, "Avatar not found", errors.New("Image not found: "+file))
}
return ctx.File(original)
return ctx.File("images/avatars/large/" + ctx.Get("file"))
})
// Avatars
app.Get("/images/avatars/small/:file", func(ctx *aero.Context) string {
file := strings.TrimSuffix(ctx.Get("file"), ".webp")
if ctx.CanUseWebP() {
return ctx.File("images/avatars/small/webp/" + file + ".webp")
}
original := arn.FindFileWithExtension(
file,
"images/avatars/small/original/",
arn.OriginalImageExtensions,
)
if original == "" {
return ctx.Error(http.StatusNotFound, "Avatar not found", errors.New("Image not found: "+file))
}
return ctx.File(original)
return ctx.File("images/avatars/small/" + ctx.Get("file"))
})
// Elements

19
auth/api-keys.go
View File

@ -1,19 +0,0 @@
package auth
import (
"encoding/json"
"io/ioutil"
"github.com/animenotifier/arn"
)
var apiKeys arn.APIKeys
func init() {
data, _ := ioutil.ReadFile("security/api-keys.json")
err := json.Unmarshal(data, &apiKeys)
if err != nil {
panic(err)
}
}

5
auth/auth.go
View File

@ -3,11 +3,16 @@ package auth
import "github.com/aerogo/aero"
import "github.com/animenotifier/notify.moe/utils"
const newUserStartRoute = "/settings"
// Install ...
func Install(app *aero.Application) {
// Google
InstallGoogleAuth(app)
// Facebook
InstallFacebookAuth(app)
// Logout
app.Get("/logout", func(ctx *aero.Context) string {
if ctx.HasSession() {

164
auth/facebook.go Normal file
View File

@ -0,0 +1,164 @@
package auth
import (
"encoding/json"
"errors"
"io/ioutil"
"net/http"
"strings"
"github.com/aerogo/aero"
"github.com/animenotifier/arn"
"github.com/animenotifier/notify.moe/utils"
"golang.org/x/oauth2"
"golang.org/x/oauth2/facebook"
)
// FacebookUser is the user data we receive from Facebook
type FacebookUser struct {
ID string `json:"id"`
Email string `json:"email"`
FirstName string `json:"first_name"`
LastName string `json:"last_name"`
Gender string `json:"gender"`
}
// InstallFacebookAuth enables Facebook login for the app.
func InstallFacebookAuth(app *aero.Application) {
config := &oauth2.Config{
ClientID: arn.APIKeys.Facebook.ID,
ClientSecret: arn.APIKeys.Facebook.Secret,
RedirectURL: "https://" + app.Config.Domain + "/auth/facebook/callback",
Scopes: []string{
"public_profile",
"email",
},
Endpoint: facebook.Endpoint,
}
// Auth
app.Get("/auth/facebook", func(ctx *aero.Context) string {
state := ctx.Session().ID()
url := config.AuthCodeURL(state)
ctx.Redirect(url)
return ""
})
// Auth Callback
app.Get("/auth/facebook/callback", func(ctx *aero.Context) string {
if !ctx.HasSession() {
return ctx.Error(http.StatusUnauthorized, "Facebook login failed", errors.New("Session does not exist"))
}
session := ctx.Session()
if session.ID() != ctx.Query("state") {
return ctx.Error(http.StatusUnauthorized, "Facebook login failed", errors.New("Incorrect state"))
}
// Handle the exchange code to initiate a transport
token, err := config.Exchange(oauth2.NoContext, ctx.Query("code"))
if err != nil {
return ctx.Error(http.StatusBadRequest, "Could not obtain OAuth token", err)
}
// Construct the OAuth client
client := config.Client(oauth2.NoContext, token)
// Fetch user data from Facebook
resp, err := client.Get("https://graph.facebook.com/me?fields=email,first_name,last_name,gender")
if err != nil {
return ctx.Error(http.StatusBadRequest, "Failed requesting user data from Facebook", err)
}
defer resp.Body.Close()
body, _ := ioutil.ReadAll(resp.Body)
// Construct a FacebookUser object
fbUser := FacebookUser{}
err = json.Unmarshal(body, &fbUser)
if err != nil {
return ctx.Error(http.StatusBadRequest, "Failed parsing user data (JSON)", err)
}
// Change googlemail.com to gmail.com
fbUser.Email = strings.Replace(fbUser.Email, "googlemail.com", "gmail.com", 1)
// Is this an existing user connecting another social account?
user := utils.GetUser(ctx)
if user != nil {
// Add FacebookToUser reference
user.ConnectFacebook(fbUser.ID)
// Save in DB
user.Save()
// Log
authLog.Info("Added Facebook ID to existing account", user.ID, user.Nick, ctx.RealIP(), user.Email, user.RealName())
return ctx.Redirect("/")
}
var getErr error
// Try to find an existing user via the Facebook user ID
user, getErr = arn.GetUserByFacebookID(fbUser.ID)
if getErr == nil && user != nil {
authLog.Info("User logged in via Facebook ID", user.ID, user.Nick, ctx.RealIP(), user.Email, user.RealName())
user.LastLogin = arn.DateTimeUTC()
user.Save()
session.Set("userId", user.ID)
return ctx.Redirect("/")
}
// Try to find an existing user via the associated e-mail address
user, getErr = arn.GetUserByEmail(fbUser.Email)
if getErr == nil && user != nil {
authLog.Info("User logged in via Email", user.ID, user.Nick, ctx.RealIP(), user.Email, user.RealName())
user.LastLogin = arn.DateTimeUTC()
user.Save()
session.Set("userId", user.ID)
return ctx.Redirect("/")
}
// Register new user
user = arn.NewUser()
user.Nick = "fb" + fbUser.ID
user.Email = fbUser.Email
user.FirstName = fbUser.FirstName
user.LastName = fbUser.LastName
user.Gender = fbUser.Gender
user.LastLogin = arn.DateTimeUTC()
// Save basic user info already to avoid data inconsistency problems
user.Save()
// Register user
arn.RegisterUser(user)
// Connect account to a Facebook account
user.ConnectFacebook(fbUser.ID)
// Save user object again with updated data
user.Save()
// Login
session.Set("userId", user.ID)
// Log
authLog.Info("Registered new user via Facebook", user.ID, user.Nick, ctx.RealIP(), user.Email, user.RealName())
// Redirect to settings
return ctx.Redirect(newUserStartRoute)
})
}

46
auth/google.go
View File

@ -5,6 +5,7 @@ import (
"errors"
"io/ioutil"
"net/http"
"strings"
"github.com/aerogo/aero"
"github.com/animenotifier/arn"
@ -29,8 +30,8 @@ type GoogleUser struct {
// InstallGoogleAuth enables Google login for the app.
func InstallGoogleAuth(app *aero.Application) {
config := &oauth2.Config{
ClientID: apiKeys.Google.ID,
ClientSecret: apiKeys.Google.Secret,
ClientID: arn.APIKeys.Google.ID,
ClientSecret: arn.APIKeys.Google.Secret,
RedirectURL: "https://" + app.Config.Domain + "/auth/google/callback",
Scopes: []string{
"https://www.googleapis.com/auth/userinfo.email",
@ -43,8 +44,8 @@ func InstallGoogleAuth(app *aero.Application) {
// Auth
app.Get("/auth/google", func(ctx *aero.Context) string {
sessionID := ctx.Session().ID()
url := config.AuthCodeURL(sessionID)
state := ctx.Session().ID()
url := config.AuthCodeURL(state)
ctx.Redirect(url)
return ""
})
@ -89,18 +90,25 @@ func InstallGoogleAuth(app *aero.Application) {
return ctx.Error(http.StatusBadRequest, "Failed parsing user data (JSON)", err)
}
if googleUser.Sub == "" {
return ctx.Error(http.StatusBadRequest, "Failed retrieving Google data", errors.New("Empty ID"))
}
// Change googlemail.com to gmail.com
googleUser.Email = strings.Replace(googleUser.Email, "googlemail.com", "gmail.com", 1)
// Is this an existing user connecting another social account?
user := utils.GetUser(ctx)
if user != nil {
println("Connected")
// Add GoogleToUser reference
err = user.ConnectGoogle(googleUser.Sub)
user.ConnectGoogle(googleUser.Sub)
if err != nil {
ctx.Error(http.StatusInternalServerError, "Could not connect account to Google account", err)
}
// Save in DB
user.Save()
// Log
authLog.Info("Added Google ID to existing account", user.ID, user.Nick, ctx.RealIP(), user.Email, user.RealName())
return ctx.Redirect("/")
}
@ -108,7 +116,7 @@ func InstallGoogleAuth(app *aero.Application) {
var getErr error
// Try to find an existing user via the Google user ID
user, getErr = arn.GetUserFromTable("GoogleToUser", googleUser.Sub)
user, getErr = arn.GetUserByGoogleID(googleUser.Sub)
if getErr == nil && user != nil {
authLog.Info("User logged in via Google ID", user.ID, user.Nick, ctx.RealIP(), user.Email, user.RealName())
@ -146,18 +154,10 @@ func InstallGoogleAuth(app *aero.Application) {
user.Save()
// Register user
err = arn.RegisterUser(user)
if err != nil {
ctx.Error(http.StatusInternalServerError, "Could not register a new user", err)
}
arn.RegisterUser(user)
// Connect account to a Google account
err = user.ConnectGoogle(googleUser.Sub)
if err != nil {
ctx.Error(http.StatusInternalServerError, "Could not connect account to Google account", err)
}
user.ConnectGoogle(googleUser.Sub)
// Save user object again with updated data
user.Save()
@ -166,9 +166,9 @@ func InstallGoogleAuth(app *aero.Application) {
session.Set("userId", user.ID)
// Log
authLog.Info("Registered new user", user.ID, user.Nick, ctx.RealIP(), user.Email, user.RealName())
authLog.Info("Registered new user via Google", user.ID, user.Nick, ctx.RealIP(), user.Email, user.RealName())
// Redirect to frontpage
return ctx.Redirect("/")
return ctx.Redirect(newUserStartRoute)
})
}

17
benchmarks/Components_test.go
View File

@ -7,7 +7,7 @@ import (
"github.com/animenotifier/notify.moe/components"
)
func BenchmarkThread(b *testing.B) {
func BenchmarkRenderThread(b *testing.B) {
thread, _ := arn.GetThread("HJgS7c2K")
thread.HTML() // Pre-render markdown
@ -25,3 +25,18 @@ func BenchmarkThread(b *testing.B) {
}
})
}
func BenchmarkRenderAnimeList(b *testing.B) {
user, _ := arn.GetUser("4J6qpK1ve")
animeList := user.AnimeList()
animeList.PrefetchAnime()
b.ReportAllocs()
b.ResetTimer()
b.RunParallel(func(pb *testing.PB) {
for pb.Next() {
components.AnimeList(animeList, user, user)
}
})
}

36
benchmarks/DB_AnimeList_test.go Normal file
View File

@ -0,0 +1,36 @@
package benchmarks
import (
"testing"
"github.com/animenotifier/arn"
)
func BenchmarkDBAnimeListGetMap(b *testing.B) {
user, _ := arn.GetUser("4J6qpK1ve")
b.ReportAllocs()
b.ResetTimer()
b.RunParallel(func(pb *testing.PB) {
for pb.Next() {
animeList, _ := arn.GetAnimeList(user.ID)
noop(animeList)
}
})
}
func BenchmarkDBAnimeListGet(b *testing.B) {
b.ReportAllocs()
b.ResetTimer()
b.RunParallel(func(pb *testing.PB) {
for pb.Next() {
list, _ := arn.DB.Get("AnimeList", "4J6qpK1ve")
animeList := list.(*arn.AnimeList)
noop(animeList)
}
})
}
func noop(list *arn.AnimeList) {}

67
bots/avatars/avatars.go Normal file
View File

@ -0,0 +1,67 @@
package main
import (
"encoding/json"
"flag"
"io"
"log"
"net/http"
"os"
"path"
"strings"
"github.com/animenotifier/arn"
_ "image/gif"
_ "image/jpeg"
_ "image/png"
"github.com/animenotifier/avatar/lib"
)
var port = "8001"
func init() {
flag.StringVar(&port, "port", "", "Port the HTTP server should listen on")
flag.Parse()
}
func main() {
// Switch to main directory
exe, err := os.Executable()
if err != nil {
panic(err)
}
root := path.Dir(exe)
os.Chdir(path.Join(root, "../../"))
// Start server
http.HandleFunc("/", onRequest)
log.Fatal(http.ListenAndServe(":"+port, nil))
}
// onRequest handles requests and refreshes the requested avatar
func onRequest(w http.ResponseWriter, req *http.Request) {
userID := strings.TrimPrefix(req.URL.Path, "/")
user, err := arn.GetUser(userID)
if err != nil {
w.WriteHeader(http.StatusInternalServerError)
io.WriteString(w, err.Error())
return
}
// Refresh
lib.RefreshAvatar(user)
// Send JSON response
buffer, err := json.Marshal(user.Avatar)
if err != nil {
io.WriteString(w, err.Error())
}
w.Write(buffer)
}

4
bots/build.sh Executable file
View File

@ -0,0 +1,4 @@
#!/bin/sh
MYDIR="$(dirname "$(realpath "$0")")"
cd "$MYDIR"
for dir in ./*; do ([ -d "$dir" ] && cd "$dir" && echo "Building bots/$dir" && go build); done

59
jobs/discord/main.go → bots/discord/discord.go
View File

@ -1,13 +1,9 @@
package main
import (
"encoding/json"
"io/ioutil"
"log"
"os"
"os/signal"
"path"
"path/filepath"
"strings"
"syscall"
@ -21,30 +17,8 @@ var discord *discordgo.Session
func main() {
var err error
exe, err := os.Executable()
if err != nil {
panic(err)
}
dir := path.Dir(exe)
var apiKeysPath string
apiKeysPath, err = filepath.Abs(dir + "/../../security/api-keys.json")
if err != nil {
panic(err)
}
var apiKeys arn.APIKeys
data, _ := ioutil.ReadFile(apiKeysPath)
err = json.Unmarshal(data, &apiKeys)
if err != nil {
panic(err)
}
discord, _ = discordgo.New()
discord.Token = "Bot " + apiKeys.Discord.Token
discord.Token = "Bot " + arn.APIKeys.Discord.Token
// Verify a Token was provided
if discord.Token == "" {
@ -93,6 +67,14 @@ func onMessage(s *discordgo.Session, m *discordgo.MessageCreate) {
**!tag** [forum tag]`)
}
// Has the bot been mentioned?
for _, user := range m.Mentions {
if user.ID == discord.State.User.ID {
s.ChannelMessageSend(m.ChannelID, m.Author.Mention()+" :heart:")
return
}
}
if strings.HasPrefix(m.Content, "!user ") {
s.ChannelMessageSend(m.ChannelID, "https://notify.moe/+"+strings.Split(m.Content, " ")[1])
return
@ -113,21 +95,34 @@ func onMessage(s *discordgo.Session, m *discordgo.MessageCreate) {
return
}
if strings.HasPrefix(m.Content, "!play ") {
s.UpdateStatus(0, strings.Split(m.Content, " ")[1])
return
}
if strings.HasPrefix(m.Content, "!s ") {
term := m.Content[len("!s "):]
userResults, animeResults := arn.Search(term, 10, 10)
users, animes, posts, threads := arn.Search(term, 3, 3, 3, 3)
message := ""
for _, user := range userResults {
for _, user := range users {
message += "https://notify.moe" + user.Link() + "\n"
}
for _, anime := range animeResults {
for _, anime := range animes {
message += "https://notify.moe" + anime.Link() + "\n"
}
if len(userResults) == 0 && len(animeResults) == 0 {
message = "Sorry, I couldn't find any anime or users with that term."
for _, post := range posts {
message += "https://notify.moe" + post.Link() + "\n"
}
for _, thread := range threads {
message += "https://notify.moe" + thread.Link() + "\n"
}
if len(users) == 0 && len(animes) == 0 && len(posts) == 0 && len(threads) == 0 {
message = "Sorry, I couldn't find anything using that term."
}
s.ChannelMessageSend(m.ChannelID, message)

24
config.json
View File

@ -6,6 +6,7 @@
],
"styles": [
"include/config",
"include/dark",
"include/mixins",
"reset",
"base",
@ -17,6 +18,7 @@
"input",
"grid",
"forum",
"tabs",
"user",
"video",
"loading",
@ -30,20 +32,28 @@
"manifest": {
"short_name": "notify.moe",
"gcm_sender_id": "941298467524",
"theme_color": "#f8a582",
"theme_color": "#f0c7bb",
"background_color": "#ffffff",
"icons": [
{
"src": "images/brand/64",
"sizes": "64x64"
"src": "images/brand/64.png",
"sizes": "64x64",
"type": "image/png"
},
{
"src": "images/brand/300",
"sizes": "300x300"
"src": "images/brand/128.png",
"sizes": "128x128",
"type": "image/png"
},
{
"src": "images/brand/600",
"sizes": "600x600"
"src": "images/brand/144.png",
"sizes": "144x144",
"type": "image/png"
},
{
"src": "images/brand/220.png",
"sizes": "220x220",
"type": "image/png"
}
]
},

BIN
images/brand/128.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 32 KiB

BIN
images/brand/128.webp Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 8.3 KiB

BIN
images/brand/144.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 36 KiB

BIN
images/brand/144.webp Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 12 KiB

BIN
images/brand/220.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 69 KiB

BIN
images/brand/220.webp Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 23 KiB

BIN
images/brand/300.png
View File

Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 125 KiB

BIN
images/brand/300.webp
View File

Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 38 KiB

BIN
images/brand/600.png
View File

Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 447 KiB

BIN
images/brand/600.webp
View File

Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 97 KiB

BIN
images/brand/64.png
View File

Binary file not shown.
Side by Side Swipe Overlay

Before

Width:  |  Height:  |  Size: 11 KiB

After

Width:  |  Height:  |  Size: 12 KiB

BIN
images/brand/64.webp
View File

Binary file not shown.
Side by Side Swipe Overlay

Before

Width:  |  Height:  |  Size: 3.8 KiB

After

Width:  |  Height:  |  Size: 3.5 KiB

BIN
images/elements/extension-screenshot.png
View File

Binary file not shown.
Side by Side Swipe Overlay

Before

Width:  |  Height:  |  Size: 50 KiB

After

Width:  |  Height:  |  Size: 37 KiB

BIN
images/elements/noise-light.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 1.3 KiB

BIN
images/elements/noise-strong.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 3.1 KiB

BIN
images/elements/thank-you.jpg Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 296 KiB

52
jobs/active-users/main.go
View File

@ -1,52 +0,0 @@
package main
import (
"fmt"
"sort"
"github.com/animenotifier/arn"
"github.com/fatih/color"
)
func main() {
color.Yellow("Caching list of active users")
cache := arn.ListOfIDs{}
// Filter out active users with an avatar
users, err := arn.FilterUsers(func(user *arn.User) bool {
return user.IsActive() && user.Avatar != ""
})
if err != nil {
panic(err)
}
// Sort
sort.Slice(users, func(i, j int) bool {
if users[i].LastSeen < users[j].LastSeen {
return false
}
if users[i].LastSeen > users[j].LastSeen {
return true
}
return users[i].Registered > users[j].Registered
})
// Add users to list
for _, user := range users {
cache.IDList = append(cache.IDList, user.ID)
}
fmt.Println(len(cache.IDList), "users")
err = arn.DB.Set("Cache", "active users", cache)
if err != nil {
panic(err)
}
color.Green("Finished.")
}

43
jobs/airing-anime/main.go
View File

@ -1,43 +0,0 @@
package main
import (
"sort"
"github.com/animenotifier/arn"
"github.com/fatih/color"
)
func main() {
color.Yellow("Caching airing anime")
animeList, err := arn.GetAiringAnime()
if err != nil {
color.Red("Failed fetching airing anime")
color.Red(err.Error())
return
}
sort.Slice(animeList, func(i, j int) bool {
return animeList[i].Rating.Overall > animeList[j].Rating.Overall
})
// Convert to small anime list
cache := &arn.ListOfIDs{}
for _, anime := range animeList {
cache.IDList = append(cache.IDList, anime.ID)
}
println(len(cache.IDList))
saveErr := arn.DB.Set("Cache", "airing anime", cache)
if saveErr != nil {
color.Red("Error saving airing anime")
color.Red(saveErr.Error())
return
}
color.Green("Finished.")
}

32
jobs/anime-characters/anime-characters.go Normal file
View File

@ -0,0 +1,32 @@
package main
import (
"fmt"
"time"
"github.com/animenotifier/arn"
"github.com/fatih/color"
)
func main() {
color.Yellow("Refreshing anime characters...")
defer arn.Node.Close()
allAnime, _ := arn.AllAnime()
rateLimiter := time.NewTicker(500 * time.Millisecond)
for _, anime := range allAnime {
<-rateLimiter.C
chars, err := anime.RefreshAnimeCharacters()
if err != nil {
color.Red(err.Error())
continue
}
fmt.Printf("%s %s (%d characters)\n", anime.ID, anime.Title.Canonical, len(chars.Items))
}
color.Green("Finished.")
}

59
jobs/anime-images/anime-images.go Normal file
View File

@ -0,0 +1,59 @@
package main
import (
"fmt"
"io/ioutil"
"net/http"
"strings"
"time"
"github.com/aerogo/flow/jobqueue"
"github.com/animenotifier/arn"
"github.com/fatih/color"
"github.com/parnurzeal/gorequest"
)
var ticker = time.NewTicker(50 * time.Millisecond)
func main() {
color.Yellow("Downloading anime images")
defer arn.Node.Close()
jobs := jobqueue.New(work)
allAnime, _ := arn.AllAnime()
for _, anime := range allAnime {
jobs.Queue(anime)
}
results := jobs.Wait()
color.Green("Finished downloading %d anime images.", len(results))
}
func work(job interface{}) interface{} {
anime := job.(*arn.Anime)
if !strings.HasPrefix(anime.Image.Original, "//media.kitsu.io/anime/") {
return nil
}
<-ticker.C
resp, body, errs := gorequest.New().Get(anime.Image.Original).End()
if len(errs) > 0 {
color.Red(errs[0].Error())
return errs[0]
}
if resp.StatusCode != http.StatusOK {
color.Red("Status %d", resp.StatusCode)
}
extension := anime.Image.Original[strings.LastIndex(anime.Image.Original, "."):]
fileName := "anime/" + anime.ID + extension
fmt.Println(fileName)
ioutil.WriteFile(fileName, []byte(body), 0644)
return nil
}

130
jobs/anime-ratings/anime-ratings.go Normal file
View File

@ -0,0 +1,130 @@
package main
import (
"github.com/animenotifier/arn"
"github.com/fatih/color"
)
var ratings = map[string][]*arn.AnimeRating{}
var finalRating = map[string]*arn.AnimeRating{}
var popularity = map[string]*arn.AnimePopularity{}
// Note this is using the airing-anime as a template with modfications
// made to it.
func main() {
color.Yellow("Updating anime ratings")
defer arn.Node.Close()
allAnimeLists, err := arn.AllAnimeLists()
arn.PanicOnError(err)
for _, animeList := range allAnimeLists {
extractRatings(animeList)
extractPopularity(animeList)
}
// Calculate rating
for animeID := range finalRating {
overall := []float64{}
story := []float64{}
visuals := []float64{}
soundtrack := []float64{}
for _, rating := range ratings[animeID] {
if rating.Overall != 0 {
overall = append(overall, rating.Overall)
}
if rating.Story != 0 {
story = append(story, rating.Story)
}
if rating.Visuals != 0 {
visuals = append(visuals, rating.Visuals)
}
if rating.Soundtrack != 0 {
soundtrack = append(soundtrack, rating.Soundtrack)
}
}
finalRating[animeID].Overall = average(overall)
finalRating[animeID].Story = average(story)
finalRating[animeID].Visuals = average(visuals)
finalRating[animeID].Soundtrack = average(soundtrack)
}
// Save
for animeID := range finalRating {
anime, err := arn.GetAnime(animeID)
arn.PanicOnError(err)
anime.Rating = finalRating[animeID]
anime.Save()
}
// Save popularity
for animeID := range popularity {
anime, err := arn.GetAnime(animeID)
arn.PanicOnError(err)
anime.Popularity = popularity[animeID]
anime.Save()
}
color.Green("Finished.")
}
func average(floatSlice []float64) float64 {
if len(floatSlice) == 0 {
return arn.DefaultAverageRating
}
var sum float64
for _, value := range floatSlice {
sum += value
}
return sum / float64(len(floatSlice))
}
func extractRatings(animeList *arn.AnimeList) {
for _, item := range animeList.Items {
if item.Rating.IsNotRated() {
continue
}
_, found := ratings[item.AnimeID]
if !found {
ratings[item.AnimeID] = []*arn.AnimeRating{}
finalRating[item.AnimeID] = &arn.AnimeRating{}
}
ratings[item.AnimeID] = append(ratings[item.AnimeID], item.Rating)
}
}
func extractPopularity(animeList *arn.AnimeList) {
for _, item := range animeList.Items {
_, found := popularity[item.AnimeID]
if !found {
popularity[item.AnimeID] = &arn.AnimePopularity{}
}
counter := popularity[item.AnimeID]
switch item.Status {
case arn.AnimeListStatusWatching:
counter.Watching++
case arn.AnimeListStatusCompleted:
counter.Completed++
case arn.AnimeListStatusPlanned:
counter.Planned++
case arn.AnimeListStatusHold:
counter.Hold++
case arn.AnimeListStatusDropped:
counter.Dropped++
}
}
}

66
jobs/avatars/Avatar.go
View File

@ -1,66 +0,0 @@
package main
import (
"bytes"
"fmt"
"image"
"net/http"
"time"
"github.com/animenotifier/arn"
"github.com/parnurzeal/gorequest"
)
var netLog = avatarLog.NewChannel("NET")
// Avatar represents a single image and the name of the format.
type Avatar struct {
User *arn.User
Image image.Image
Data []byte
Format string
}
// String returns a text representation of the format, width and height.
func (avatar *Avatar) String() string {
return fmt.Sprint(avatar.Format, " | ", avatar.Image.Bounds().Dx(), "x", avatar.Image.Bounds().Dy())
}
// AvatarFromURL downloads and decodes the image from an URL and creates an Avatar.
func AvatarFromURL(url string, user *arn.User) *Avatar {
// Download
response, data, networkErr := gorequest.New().Get(url).EndBytes()
// Retry after 5 seconds if service unavailable
if response.StatusCode == http.StatusServiceUnavailable {
time.Sleep(5 * time.Second)
response, data, networkErr = gorequest.New().Get(url).EndBytes()
}
// Network errors
if networkErr != nil {
netLog.Error(user.Nick, url, networkErr)
return nil
}
// Bad status codes
if response.StatusCode != http.StatusOK {
netLog.Error(user.Nick, url, response.StatusCode)
return nil
}
// Decode
img, format, decodeErr := image.Decode(bytes.NewReader(data))
if decodeErr != nil {
netLog.Error(user.Nick, url, decodeErr)
return nil
}
return &Avatar{
User: user,
Image: img,
Data: data,
Format: format,
}
}

64
jobs/avatars/AvatarOriginalFileOutput.go
View File

@ -1,64 +0,0 @@
package main
import (
"bytes"
"errors"
"image/gif"
"image/jpeg"
"image/png"
"io/ioutil"
"github.com/nfnt/resize"
)
// AvatarOriginalFileOutput ...
type AvatarOriginalFileOutput struct {
Directory string
Size int
}
// SaveAvatar writes the original avatar to the file system.
func (output *AvatarOriginalFileOutput) SaveAvatar(avatar *Avatar) error {
// Determine file extension
extension := ""
switch avatar.Format {
case "jpg", "jpeg":
extension = ".jpg"
case "png":
extension = ".png"
case "gif":
extension = ".gif"
default:
return errors.New("Unknown format: " + avatar.Format)
}
// Resize if needed
data := avatar.Data
img := avatar.Image
if img.Bounds().Dx() > output.Size {
img = resize.Resize(uint(output.Size), 0, img, resize.Lanczos3)
buffer := new(bytes.Buffer)
var err error
switch extension {
case ".jpg":
err = jpeg.Encode(buffer, img, nil)
case ".png":
err = png.Encode(buffer, img)
case ".gif":
err = gif.Encode(buffer, img, nil)
}
if err != nil {
return err
}
data = buffer.Bytes()
}
// Write to file
fileName := output.Directory + avatar.User.ID + extension
return ioutil.WriteFile(fileName, data, 0644)
}

10
jobs/avatars/AvatarSource.go
View File

@ -1,10 +0,0 @@
package main
import (
"github.com/animenotifier/arn"
)
// AvatarSource describes a source where we can find avatar images for a user.
type AvatarSource interface {
GetAvatar(*arn.User) *Avatar
}

27
jobs/avatars/AvatarWebPFileOutput.go
View File

@ -1,27 +0,0 @@
package main
import (
"github.com/animenotifier/arn"
"github.com/nfnt/resize"
)
// AvatarWebPFileOutput ...
type AvatarWebPFileOutput struct {
Directory string
Size int
Quality float32
}
// SaveAvatar writes the avatar in WebP format to the file system.
func (output *AvatarWebPFileOutput) SaveAvatar(avatar *Avatar) error {
img := avatar.Image
// Resize if needed
if img.Bounds().Dx() > output.Size {
img = resize.Resize(uint(output.Size), 0, img, resize.Lanczos3)
}
// Write to file
fileName := output.Directory + avatar.User.ID + ".webp"
return arn.SaveWebP(img, fileName, output.Quality)
}

6
jobs/avatars/AvatarWriter.go
View File

@ -1,6 +0,0 @@
package main
// AvatarOutput represents a system that saves an avatar locally (in database or as a file, e.g.)
type AvatarOutput interface {
SaveAvatar(*Avatar) error
}

35
jobs/avatars/Gravatar.go
View File

@ -1,35 +0,0 @@
package main
import (
"fmt"
"time"
"github.com/animenotifier/arn"
gravatar "github.com/ungerik/go-gravatar"
)
var gravatarLog = avatarLog.NewChannel("GRA")
// Gravatar - https://gravatar.com/
type Gravatar struct {
Rating string
RequestLimiter *time.Ticker
}
// GetAvatar returns the Gravatar image for a user (if available).
func (source *Gravatar) GetAvatar(user *arn.User) *Avatar {
// If the user has no Email registered we can't get a Gravatar.
if user.Email == "" {
gravatarLog.Error(user.Nick, "No Email")
return nil
}
// Build URL
gravatarURL := gravatar.Url(user.Email) + "?s=" + fmt.Sprint(arn.AvatarMaxSize) + "&d=404&r=" + source.Rating
// Wait for request limiter to allow us to send a request
<-source.RequestLimiter.C
// Download
return AvatarFromURL(gravatarURL, user)
}

60
jobs/avatars/MyAnimeList.go
View File

@ -1,60 +0,0 @@
package main
import (
"net/http"
"regexp"
"time"
"github.com/animenotifier/arn"
"github.com/parnurzeal/gorequest"
)
var userIDRegex = regexp.MustCompile(`<user_id>(\d+)<\/user_id>`)
var malLog = avatarLog.NewChannel("MAL")
// MyAnimeList - https://myanimelist.net/
type MyAnimeList struct {
RequestLimiter *time.Ticker
}
// GetAvatar returns the Gravatar image for a user (if available).
func (source *MyAnimeList) GetAvatar(user *arn.User) *Avatar {
malNick := user.Accounts.MyAnimeList.Nick
// If the user has no username we can't get an avatar.
if malNick == "" {
malLog.Error(user.Nick, "No MAL nick")
return nil
}
// Download user info
userInfoURL := "https://myanimelist.net/malappinfo.php?u=" + malNick
response, xml, networkErr := gorequest.New().Get(userInfoURL).End()
if networkErr != nil {
malLog.Error(user.Nick, userInfoURL, networkErr)
return nil
}
if response.StatusCode != http.StatusOK {
malLog.Error(user.Nick, userInfoURL, response.StatusCode)
return nil
}
// Build URL
matches := userIDRegex.FindStringSubmatch(xml)
if matches == nil || len(matches) < 2 {
malLog.Error(user.Nick, "Could not find user ID")
return nil
}
malID := matches[1]
malAvatarURL := "https://myanimelist.cdn-dena.com/images/userimages/" + malID + ".jpg"
// Wait for request limiter to allow us to send a request
<-source.RequestLimiter.C
// Download
return AvatarFromURL(malAvatarURL, user)
}

69
jobs/avatars/avatars.go Normal file
View File

@ -0,0 +1,69 @@
package main
import (
"os"
"path"
"runtime"
"sync"
_ "image/gif"
_ "image/jpeg"
_ "image/png"
"github.com/aerogo/log"
"github.com/animenotifier/arn"
"github.com/animenotifier/avatar/lib"
"github.com/fatih/color"
)
var wg sync.WaitGroup
// Main
func main() {
color.Yellow("Generating user avatars")
defer arn.Node.Close()
// Switch to main directory
exe, err := os.Executable()
if err != nil {
panic(err)
}
root := path.Dir(exe)
os.Chdir(path.Join(root, "../../"))
// Log
lib.Log.AddOutput(log.File("logs/avatar.log"))
defer lib.Log.Flush()
if InvokeShellArgs() {
return
}
// Worker queue
usersQueue := make(chan *arn.User, runtime.NumCPU())
StartWorkers(usersQueue, lib.RefreshAvatar)
// We'll send each user to one of the worker threads
for user := range arn.StreamUsers() {
wg.Add(1)
usersQueue <- user
}
wg.Wait()
color.Green("Finished.")
}
// StartWorkers creates multiple workers to handle a user each.
func StartWorkers(queue chan *arn.User, work func(*arn.User)) {
for w := 0; w < runtime.NumCPU(); w++ {
go func() {
for user := range queue {
work(user)
wg.Done()
}
}()
}
}

148
jobs/avatars/main.go
View File

@ -1,148 +0,0 @@
package main
import (
"fmt"
"os"
"path"
"reflect"
"runtime"
"time"
_ "image/gif"
_ "image/jpeg"
_ "image/png"
"github.com/aerogo/log"
"github.com/animenotifier/arn"
"github.com/fatih/color"
)
const (
webPQuality = 80
)
var avatarSources []AvatarSource
var avatarOutputs []AvatarOutput
var avatarLog = log.New()
// Main
func main() {
color.Yellow("Generating user avatars")
// Switch to main directory
exe, err := os.Executable()
if err != nil {
panic(err)
}
root := path.Dir(exe)
os.Chdir(path.Join(root, "../../"))
// Log
avatarLog.AddOutput(log.File("logs/avatar.log"))
defer avatarLog.Flush()
// Define the avatar sources
avatarSources = []AvatarSource{
&Gravatar{
Rating: "pg",
RequestLimiter: time.NewTicker(250 * time.Millisecond),
},
&MyAnimeList{
RequestLimiter: time.NewTicker(250 * time.Millisecond),
},
}
// Define the avatar outputs
avatarOutputs = []AvatarOutput{
// Original - Large
&AvatarOriginalFileOutput{
Directory: "images/avatars/large/original/",
Size: arn.AvatarMaxSize,
},
// Original - Small
&AvatarOriginalFileOutput{
Directory: "images/avatars/small/original/",
Size: arn.AvatarSmallSize,
},
// WebP - Large
&AvatarWebPFileOutput{
Directory: "images/avatars/large/webp/",
Size: arn.AvatarMaxSize,
Quality: webPQuality,
},
// WebP - Small
&AvatarWebPFileOutput{
Directory: "images/avatars/small/webp/",
Size: arn.AvatarSmallSize,
Quality: webPQuality,
},
}
if InvokeShellArgs() {
return
}
// Stream of all users
users, _ := arn.FilterUsers(func(user *arn.User) bool {
return true
})
// Log user count
println(len(users), "users")
// Worker queue
usersQueue := make(chan *arn.User)
StartWorkers(usersQueue, Work)
// We'll send each user to one of the worker threads
for _, user := range users {
usersQueue <- user
}
color.Green("Finished.")
}
// StartWorkers creates multiple workers to handle a user each.
func StartWorkers(queue chan *arn.User, work func(*arn.User)) {
for w := 0; w < runtime.NumCPU(); w++ {
go func() {
for user := range queue {
work(user)
}
}()
}
}
// Work handles a single user.
func Work(user *arn.User) {
user.Avatar = ""
for _, source := range avatarSources {
avatar := source.GetAvatar(user)
if avatar == nil {
// fmt.Println(color.RedString("✘"), reflect.TypeOf(source).Elem().Name(), user.Nick)
continue
}
for _, writer := range avatarOutputs {
err := writer.SaveAvatar(avatar)
if err != nil {
color.Red(err.Error())
}
}
fmt.Println(color.GreenString("✔"), reflect.TypeOf(source).Elem().Name(), "|", user.Nick, "|", avatar)
user.Avatar = "/+" + user.Nick + "/avatar"
break
}
// Save avatar data
user.Save()
}

5
jobs/avatars/shell.go
View File

@ -4,6 +4,7 @@ import (
"flag"
"github.com/animenotifier/arn"
"github.com/animenotifier/avatar/lib"
)
// Shell parameters
@ -26,7 +27,7 @@ func InvokeShellArgs() bool {
panic(err)
}
Work(user)
lib.RefreshAvatar(user)
return true
}
@ -37,7 +38,7 @@ func InvokeShellArgs() bool {
panic(err)
}
Work(user)
lib.RefreshAvatar(user)
return true
}

1
jobs/build.sh
View File

@ -1,4 +1,5 @@
#!/bin/sh
MYDIR="$(dirname "$(realpath "$0")")"
cd "$MYDIR"
go build
for dir in ./*; do ([ -d "$dir" ] && cd "$dir" && echo "Building jobs/$dir" && go build); done

15
jobs/main.go → jobs/jobs.go
View File

@ -23,12 +23,15 @@ var colorPool = []*color.Color{
}
var jobs = map[string]time.Duration{
"active-users": 1 * time.Minute,
"avatars": 1 * time.Hour,
"sync-anime": 10 * time.Hour,
"popular-anime": 11 * time.Hour,
"airing-anime": 12 * time.Hour,
"search-index": 13 * time.Hour,
"anime-ratings": 10 * time.Minute,
"avatars": 1 * time.Hour,
"test": 1 * time.Hour,
"twist": 2 * time.Hour,
"search-index": 2 * time.Hour,
"refresh-episodes": 10 * time.Hour,
"refresh-osu": 12 * time.Hour,
"sync-anime": 12 * time.Hour,
"sync-shoboi": 24 * time.Hour,
}
func main() {

56
jobs/popular-anime/main.go
View File

@ -1,56 +0,0 @@
package main
import (
"sort"
"github.com/animenotifier/arn"
"github.com/fatih/color"
)
const maxPopularAnime = 10
// Note this is using the airing-anime as a template with modfications
// made to it.
func main() {
color.Yellow("Caching popular anime")
animeChan, err := arn.AllAnime()
if err != nil {
color.Red("Failed fetching anime channel")
color.Red(err.Error())
return
}
var animeList []*arn.Anime
for anime := range animeChan {
animeList = append(animeList, anime)
}
sort.Slice(animeList, func(i, j int) bool {
return animeList[i].Rating.Overall > animeList[j].Rating.Overall
})
// Change size of anime list to 10
animeList = animeList[:maxPopularAnime]
// Convert to small anime list
cache := &arn.ListOfIDs{}
for _, anime := range animeList {
cache.IDList = append(cache.IDList, anime.ID)
}
println(len(cache.IDList))
saveErr := arn.DB.Set("Cache", "popular anime", cache)
if saveErr != nil {
color.Red("Error saving popular anime")
color.Red(saveErr.Error())
return
}
color.Green("Finished.")
}

79
jobs/refresh-episodes/refresh-episodes.go Normal file
View File

@ -0,0 +1,79 @@
package main
import (
"fmt"
"strings"
"github.com/animenotifier/arn"
"github.com/fatih/color"
)
func main() {
color.Yellow("Refreshing episode information for each anime.")
defer arn.Node.Close()
if InvokeShellArgs() {
return
}
highPriority := []*arn.Anime{}
mediumPriority := []*arn.Anime{}
lowPriority := []*arn.Anime{}
for anime := range arn.StreamAnime() {
if anime.GetMapping("shoboi/anime") == "" {
continue
}
// The rest gets sorted by airing status
switch anime.Status {
case "current":
highPriority = append(highPriority, anime)
case "upcoming":
mediumPriority = append(mediumPriority, anime)
default:
lowPriority = append(lowPriority, anime)
}
}
color.Cyan("High priority queue (%d):", len(highPriority))
refreshQueue(highPriority)
color.Cyan("Medium priority queue (%d):", len(mediumPriority))
refreshQueue(mediumPriority)
color.Cyan("Low priority queue (%d):", len(lowPriority))
refreshQueue(lowPriority)
color.Green("Finished.")
}
func refreshQueue(queue []*arn.Anime) {
for _, anime := range queue {
refresh(anime)
}
}
func refresh(anime *arn.Anime) {
fmt.Println(anime.ID, "|", anime.Title.Canonical, "|", anime.GetMapping("shoboi/anime"))
episodeCount := len(anime.Episodes().Items)
availableEpisodeCount := anime.Episodes().AvailableCount()
err := anime.RefreshEpisodes()
if err != nil {
if strings.Contains(err.Error(), "missing a Shoboi ID") {
return
}
color.Red(err.Error())
} else {
faint := color.New(color.Faint).SprintFunc()
episodes := anime.Episodes()
fmt.Println(faint(episodes))
fmt.Printf("+%d episodes | +%d available (%d total)\n", len(episodes.Items)-episodeCount, episodes.AvailableCount()-availableEpisodeCount, len(episodes.Items))
println()
}
}

32
jobs/refresh-episodes/shell.go Normal file
View File

@ -0,0 +1,32 @@
package main
import (
"flag"
"github.com/animenotifier/arn"
)
// Shell parameters
var animeID string
// Shell flags
func init() {
flag.StringVar(&animeID, "id", "", "ID of the anime you want to refresh")
flag.Parse()
}
// InvokeShellArgs ...
func InvokeShellArgs() bool {
if animeID != "" {
anime, err := arn.GetAnime(animeID)
if err != nil {
panic(err)
}
refresh(anime)
return true
}
return false
}

32
jobs/refresh-osu/refresh-osu.go Normal file
View File

@ -0,0 +1,32 @@
package main
import (
"time"
"github.com/animenotifier/arn"
"github.com/fatih/color"
)
func main() {
color.Yellow("Refreshing osu information")
defer arn.Node.Close()
ticker := time.NewTicker(500 * time.Millisecond)
for user := range arn.StreamUsers() {
// Get osu info
if user.RefreshOsuInfo() == nil {
arn.PrettyPrint(user.Accounts.Osu)
// Fetch user again to prevent writing old data
updatedUser, _ := arn.GetUser(user.ID)
updatedUser.Accounts.Osu = user.Accounts.Osu
updatedUser.Save()
}
// Wait for rate limiter
<-ticker.C
}
color.Green("Finished.")
}

74
jobs/search-index/main.go
View File

@ -1,74 +0,0 @@
package main
import (
"fmt"
"strings"
"github.com/aerogo/flow"
"github.com/animenotifier/arn"
"github.com/fatih/color"
)
func main() {
color.Yellow("Updating search index")
flow.Parallel(updateAnimeIndex, updateUserIndex)
color.Green("Finished.")
}
func updateAnimeIndex() {
animeSearchIndex := arn.NewSearchIndex()
// Anime
animeStream, err := arn.AllAnime()
if err != nil {
panic(err)
}
for anime := range animeStream {
if anime.Title.Canonical != "" {
animeSearchIndex.TextToID[strings.ToLower(anime.Title.Canonical)] = anime.ID
}
if anime.Title.Japanese != "" {
animeSearchIndex.TextToID[anime.Title.Japanese] = anime.ID
}
}
fmt.Println(len(animeSearchIndex.TextToID), "anime titles")
// Save in database
err = arn.DB.Set("SearchIndex", "Anime", animeSearchIndex)
if err != nil {
panic(err)
}
}
func updateUserIndex() {
userSearchIndex := arn.NewSearchIndex()
// Users
userStream, err := arn.AllUsers()
if err != nil {
panic(err)
}
for user := range userStream {
if user.IsActive() && user.Nick != "" {
userSearchIndex.TextToID[strings.ToLower(user.Nick)] = user.ID
}
}
fmt.Println(len(userSearchIndex.TextToID), "user names")
// Save in database
err = arn.DB.Set("SearchIndex", "User", userSearchIndex)
if err != nil {
panic(err)
}
}

108
jobs/search-index/search-index.go Normal file
View File

@ -0,0 +1,108 @@
package main
import (
"fmt"
"strings"
"github.com/aerogo/flow"
"github.com/animenotifier/arn"
"github.com/fatih/color"
)
func main() {
color.Yellow("Updating search index")
defer arn.Node.Close()
flow.Parallel(
updateAnimeIndex,
updateUserIndex,
updatePostIndex,
updateThreadIndex,
)
color.Green("Finished.")
}
func updateAnimeIndex() {
animeSearchIndex := arn.NewSearchIndex()
// Anime
for anime := range arn.StreamAnime() {
if anime.Title.Canonical != "" {
animeSearchIndex.TextToID[strings.ToLower(anime.Title.Canonical)] = anime.ID
}
if anime.Title.Romaji != "" {
animeSearchIndex.TextToID[strings.ToLower(anime.Title.Romaji)] = anime.ID
}
// Make sure we only include Japanese titles that
// don't overlap with the English titles.
if anime.Title.Japanese != "" && animeSearchIndex.TextToID[strings.ToLower(anime.Title.Japanese)] == "" {
animeSearchIndex.TextToID[strings.ToLower(anime.Title.Japanese)] = anime.ID
}
// Same with English titles, don't overwrite other stuff.
if anime.Title.English != "" && animeSearchIndex.TextToID[strings.ToLower(anime.Title.English)] == "" {
animeSearchIndex.TextToID[strings.ToLower(anime.Title.English)] = anime.ID
}
for _, synonym := range anime.Title.Synonyms {
synonym = strings.ToLower(synonym)
if synonym != "" && len(synonym) <= 10 {
animeSearchIndex.TextToID[synonym] = anime.ID
}
}
}
fmt.Println(len(animeSearchIndex.TextToID), "anime titles")
// Save in database
arn.DB.Set("SearchIndex", "Anime", animeSearchIndex)
}
func updateUserIndex() {
userSearchIndex := arn.NewSearchIndex()
// Users
for user := range arn.StreamUsers() {
if user.HasNick() {
userSearchIndex.TextToID[strings.ToLower(user.Nick)] = user.ID
}
}
fmt.Println(len(userSearchIndex.TextToID), "user names")
// Save in database
arn.DB.Set("SearchIndex", "User", userSearchIndex)
}
func updatePostIndex() {
postSearchIndex := arn.NewSearchIndex()
// Users
for post := range arn.StreamPosts() {
postSearchIndex.TextToID[strings.ToLower(post.Text)] = post.ID
}
fmt.Println(len(postSearchIndex.TextToID), "posts")
// Save in database
arn.DB.Set("SearchIndex", "Post", postSearchIndex)
}
func updateThreadIndex() {
threadSearchIndex := arn.NewSearchIndex()
// Users
for thread := range arn.StreamThreads() {
threadSearchIndex.TextToID[strings.ToLower(thread.Title)] = thread.ID
threadSearchIndex.TextToID[strings.ToLower(thread.Text)] = thread.ID
}
fmt.Println(len(threadSearchIndex.TextToID)/2, "threads")
// Save in database
arn.DB.Set("SearchIndex", "Thread", threadSearchIndex)
}

88
jobs/sync-anime/main.go
View File

@ -1,88 +0,0 @@
package main
import (
"encoding/json"
"fmt"
"strconv"
"strings"
"github.com/animenotifier/arn"
"github.com/animenotifier/kitsu"
"github.com/fatih/color"
)
func main() {
color.Yellow("Syncing Anime")
// Get a stream of all anime
allAnime := kitsu.AllAnime()
// Iterate over the stream
for anime := range allAnime {
sync(anime)
}
println("Finished.")
}
func sync(data *kitsu.Anime) {
anime := arn.Anime{}
attr := data.Attributes
// General data
anime.ID = data.ID
anime.Type = strings.ToLower(attr.ShowType)
anime.Title.Canonical = attr.CanonicalTitle
anime.Title.English = attr.Titles.En
anime.Title.Japanese = attr.Titles.JaJp
anime.Title.Romaji = attr.Titles.EnJp
anime.Title.Synonyms = attr.AbbreviatedTitles
anime.Image.Tiny = kitsu.FixImageURL(attr.PosterImage.Tiny)
anime.Image.Small = kitsu.FixImageURL(attr.PosterImage.Small)
anime.Image.Large = kitsu.FixImageURL(attr.PosterImage.Large)
anime.Image.Original = kitsu.FixImageURL(attr.PosterImage.Original)
anime.StartDate = attr.StartDate
anime.EndDate = attr.EndDate
anime.EpisodeCount = attr.EpisodeCount
anime.EpisodeLength = attr.EpisodeLength
anime.Status = attr.Status
anime.NSFW = attr.Nsfw
anime.Summary = arn.FixAnimeDescription(attr.Synopsis)
// Rating
overall, convertError := strconv.ParseFloat(attr.AverageRating, 64)
if convertError != nil {
overall = 0
}
anime.Rating.Overall = overall
// Trailers
anime.Trailers = []arn.AnimeTrailer{}
if attr.YoutubeVideoID != "" {
anime.Trailers = append(anime.Trailers, arn.AnimeTrailer{
Service: "Youtube",
VideoID: attr.YoutubeVideoID,
})
}
// Save in database
err := anime.Save()
status := ""
if err == nil {
status = color.GreenString("✔")
} else {
color.Red(err.Error())
data, _ := json.MarshalIndent(anime, "", "\t")
fmt.Println(string(data))
status = color.RedString("✘")
}
// Log
fmt.Println(status, anime.ID, anime.Title.Canonical)
}

50
jobs/sync-anime/shell.go Normal file
View File

@ -0,0 +1,50 @@
package main
import (
"errors"
"flag"
"github.com/animenotifier/arn"
"github.com/animenotifier/kitsu"
"github.com/fatih/color"
)
// Shell parameters
var animeID string
var verbose bool
// Shell flags
func init() {
flag.StringVar(&animeID, "id", "", "ID of the anime that you want to refresh")
flag.BoolVar(&verbose, "v", false, "Verbose output")
flag.Parse()
}
// InvokeShellArgs ...
func InvokeShellArgs() bool {
if animeID != "" {
kitsuAnime, err := kitsu.GetAnime(animeID)
if err != nil {
panic(err)
}
if kitsuAnime.ID != animeID {
panic(errors.New("Anime ID is not the same"))
}
anime := sync(kitsuAnime)
if verbose {
color.Cyan("Kitsu:")
arn.PrettyPrint(kitsuAnime)
color.Cyan("ARN:")
arn.PrettyPrint(anime)
}
return true
}
return false
}

143
jobs/sync-anime/sync-anime.go Normal file
View File

@ -0,0 +1,143 @@
package main
import (
"fmt"
"strings"
"github.com/animenotifier/arn"
"github.com/animenotifier/kitsu"
"github.com/fatih/color"
)
func main() {
color.Yellow("Syncing Anime")
defer arn.Node.Close()
// In case we refresh only one anime
if InvokeShellArgs() {
color.Green("Finished.")
return
}
// Get a stream of all anime
allAnime := kitsu.StreamAnimeWithMappings()
// Iterate over the stream
for anime := range allAnime {
sync(anime)
}
color.Green("Finished.")
}
func sync(data *kitsu.Anime) *arn.Anime {
anime, err := arn.GetAnime(data.ID)
if err != nil {
if strings.Contains(err.Error(), "not found") {
anime = &arn.Anime{
Title: &arn.AnimeTitle{},
Image: &arn.AnimeImageTypes{},
}
} else {
panic(err)
}
}
attr := data.Attributes
// General data
anime.ID = data.ID
anime.Type = strings.ToLower(attr.ShowType)
anime.Title.Canonical = attr.CanonicalTitle
anime.Title.English = attr.Titles.En
anime.Title.Romaji = attr.Titles.EnJp
anime.Title.Synonyms = attr.AbbreviatedTitles
anime.Image.Tiny = kitsu.FixImageURL(attr.PosterImage.Tiny)
anime.Image.Small = kitsu.FixImageURL(attr.PosterImage.Small)
anime.Image.Large = kitsu.FixImageURL(attr.PosterImage.Large)
anime.Image.Original = kitsu.FixImageURL(attr.PosterImage.Original)
anime.StartDate = attr.StartDate
anime.EndDate = attr.EndDate
anime.EpisodeCount = attr.EpisodeCount
anime.EpisodeLength = attr.EpisodeLength
anime.Status = attr.Status
anime.Summary = arn.FixAnimeDescription(attr.Synopsis)
if anime.Mappings == nil {
anime.Mappings = []*arn.Mapping{}
}
// Prefer Shoboi Japanese titles over Kitsu JP titles
if anime.GetMapping("shoboi/anime") != "" {
// Only take Kitsu title when our JP title is empty
if anime.Title.Japanese == "" {
anime.Title.Japanese = attr.Titles.JaJp
}
} else {
// Update JP title with Kitsu JP title
anime.Title.Japanese = attr.Titles.JaJp
}
// Import mappings
for _, mapping := range data.Mappings {
switch mapping.Attributes.ExternalSite {
case "myanimelist/anime":
anime.AddMapping("myanimelist/anime", mapping.Attributes.ExternalID, "")
case "anidb":
anime.AddMapping("anidb/anime", mapping.Attributes.ExternalID, "")
case "thetvdb/series":
anime.AddMapping("thetvdb/anime", mapping.Attributes.ExternalID, "")
case "thetvdb/season":
// Ignore
default:
color.Yellow("Unknown mapping: %s %s", mapping.Attributes.ExternalSite, mapping.Attributes.ExternalID)
}
}
// NSFW
if attr.Nsfw {
anime.NSFW = 1
} else {
anime.NSFW = 0
}
// Rating
if anime.Rating == nil {
anime.Rating = &arn.AnimeRating{}
}
if anime.Rating.IsNotRated() {
anime.Rating.Reset()
}
// Popularity
if anime.Popularity == nil {
anime.Popularity = &arn.AnimePopularity{}
}
// Trailers
anime.Trailers = []*arn.ExternalMedia{}
if attr.YoutubeVideoID != "" {
anime.Trailers = append(anime.Trailers, &arn.ExternalMedia{
Service: "Youtube",
ServiceID: attr.YoutubeVideoID,
})
}
// Save in database
anime.Save()
// Episodes
episodes, err := arn.GetAnimeEpisodes(anime.ID)
if err != nil || episodes == nil {
anime.RefreshEpisodes()
}
// Log
fmt.Println(color.GreenString("✔"), anime.ID, anime.Title.Canonical)
return anime
}

31
jobs/sync-characters/sync-characters.go Normal file
View File

@ -0,0 +1,31 @@
package main
import (
"fmt"
"github.com/animenotifier/arn"
"github.com/animenotifier/kitsu"
"github.com/fatih/color"
)
func main() {
color.Yellow("Syncing characters with Kitsu DB")
defer arn.Node.Close()
kitsuCharacters := kitsu.StreamCharacters()
for kitsuCharacter := range kitsuCharacters {
character := &arn.Character{
ID: kitsuCharacter.ID,
Name: kitsuCharacter.Attributes.Name,
Image: kitsu.FixImageURL(kitsuCharacter.Attributes.Image.Original),
Description: arn.FixAnimeDescription(kitsuCharacter.Attributes.Description),
}
fmt.Printf("%s %s\n", character.ID, character.Name)
character.Save()
}
color.Green("Finished.")
}

75
jobs/sync-media-relations/sync-media-relations.go Normal file
View File

@ -0,0 +1,75 @@
package main
import (
"fmt"
"strings"
"github.com/animenotifier/arn"
"github.com/animenotifier/kitsu"
"github.com/fatih/color"
)
func main() {
color.Yellow("Syncing media relations with Kitsu DB")
defer arn.Node.Close()
kitsuMediaRelations := kitsu.StreamMediaRelations()
relations := map[string]*arn.AnimeRelations{}
for mediaRelation := range kitsuMediaRelations {
// We only care about anime for now
if mediaRelation.Relationships.Source.Data.Type != "anime" || mediaRelation.Relationships.Destination.Data.Type != "anime" {
continue
}
relationType := strings.Replace(mediaRelation.Attributes.Role, "_", " ", -1)
animeID := mediaRelation.Relationships.Source.Data.ID
destinationAnimeID := mediaRelation.Relationships.Destination.Data.ID
// Confirm that the anime IDs are valid
if !arn.DB.Exists("Anime", animeID) {
continue
}
if !arn.DB.Exists("Anime", destinationAnimeID) {
continue
}
fmt.Printf(
"%s %s has %s which is %s %s\n",
mediaRelation.Relationships.Source.Data.Type,
animeID,
color.GreenString(relationType),
mediaRelation.Relationships.Destination.Data.Type,
destinationAnimeID,
)
// Add anime to the global map
relationsList, found := relations[animeID]
if !found {
relationsList = &arn.AnimeRelations{
AnimeID: animeID,
Items: []*arn.AnimeRelation{},
}
relations[animeID] = relationsList
}
relationsList.Items = append(relationsList.Items, &arn.AnimeRelation{
AnimeID: destinationAnimeID,
Type: relationType,
})
// for _, item := range relationsList.Items {
// fmt.Println("*", item.Type, item.AnimeID)
// }
}
// Save relations map
for _, animeRelations := range relations {
animeRelations.Save()
}
color.Green("Finished.")
}

120
jobs/sync-shoboi/sync-shoboi.go Normal file
View File

@ -0,0 +1,120 @@
package main
import (
"time"
"github.com/animenotifier/arn"
"github.com/animenotifier/shoboi"
"github.com/fatih/color"
)
func main() {
color.Yellow("Syncing Shoboi Anime")
defer arn.Node.Close()
// Priority queues
highPriority := []*arn.Anime{}
mediumPriority := []*arn.Anime{}
lowPriority := []*arn.Anime{}
for anime := range arn.StreamAnime() {
if anime.GetMapping("shoboi/anime") != "" {
continue
}
switch anime.Status {
case "current":
highPriority = append(highPriority, anime)
case "upcoming":
mediumPriority = append(mediumPriority, anime)
default:
lowPriority = append(lowPriority, anime)
}
}
color.Cyan("High priority queue (%d):", len(highPriority))
refreshQueue(highPriority)
color.Cyan("Medium priority queue (%d):", len(mediumPriority))
refreshQueue(mediumPriority)
color.Cyan("Low priority queue (%d):", len(lowPriority))
refreshQueue(lowPriority)
// This is a lazy hack: Wait 5 minutes for goroutines to finish their remaining work.
time.Sleep(5 * time.Minute)
color.Green("Finished.")
}
func refreshQueue(queue []*arn.Anime) {
count := 0
for _, anime := range queue {
if sync(anime) {
anime.Save()
count++
}
}
color.Green("Added Shoboi IDs for %d anime", count)
}
func sync(anime *arn.Anime) bool {
// If we already have the ID, nothing to do here
if anime.GetMapping("shoboi/anime") != "" {
return false
}
// Log ID and title
print(anime.ID + " | [JP] " + anime.Title.Japanese + " | [EN] " + anime.Title.Canonical)
// Search Japanese title
if anime.GetMapping("shoboi/anime") == "" && anime.Title.Japanese != "" {
search(anime, anime.Title.Japanese)
}
// Search English title
if anime.GetMapping("shoboi/anime") == "" && anime.Title.English != "" {
search(anime, anime.Title.English)
}
// Did we get the ID?
if anime.GetMapping("shoboi/anime") != "" {
println(color.GreenString("✔"))
return true
}
println(color.RedString("✘"))
return false
}
// Search for a specific title
func search(anime *arn.Anime, title string) {
shoboi, err := shoboi.SearchAnime(title)
if err != nil {
color.Red(err.Error())
return
}
if shoboi == nil {
return
}
// Copy titles
if shoboi.TitleJapanese != "" {
anime.Title.Japanese = shoboi.TitleJapanese
}
if shoboi.TitleHiragana != "" {
anime.Title.Hiragana = shoboi.TitleHiragana
}
if shoboi.FirstChannel != "" {
anime.FirstChannel = shoboi.FirstChannel
}
// This will start a goroutine that saves the anime
anime.AddMapping("shoboi/anime", shoboi.TID, "")
}

68
jobs/test/test.go Normal file
View File

@ -0,0 +1,68 @@
package main
import (
"os/exec"
"sync"
"github.com/animenotifier/arn"
"github.com/fatih/color"
)
var packages = []string{
"github.com/animenotifier/notify.moe",
"github.com/animenotifier/arn",
"github.com/animenotifier/kitsu",
"github.com/animenotifier/anilist",
"github.com/animenotifier/mal",
"github.com/animenotifier/shoboi",
"github.com/animenotifier/twist",
"github.com/animenotifier/avatar",
// "github.com/animenotifier/japanese",
// "github.com/animenotifier/osu",
}
func main() {
wg := sync.WaitGroup{}
for _, pkg := range packages {
wg.Add(1)
go func(pkgLocal string) {
testPackage(pkgLocal)
wg.Done()
}(pkg)
}
wg.Wait()
}
func testPackage(pkg string) {
cmd := exec.Command("go", "test", pkg+"/...")
// cmd.Stdout = os.Stdout
// cmd.Stderr = os.Stderr
err := cmd.Start()
if err != nil {
panic(err)
}
err = cmd.Wait()
if err != nil {
color.Red("%s", pkg)
// Send notification to the admin
admin, _ := arn.GetUser("4J6qpK1ve")
admin.SendNotification(&arn.Notification{
Title: pkg,
Message: "Test failed",
Link: "https://" + pkg,
Icon: "https://notify.moe/images/brand/220.png",
})
return
}
color.Green("%s", pkg)
}

48
jobs/twist/twist.go Normal file
View File

@ -0,0 +1,48 @@
package main
import (
"fmt"
"os"
"time"
"github.com/animenotifier/arn"
"github.com/animenotifier/twist"
"github.com/fatih/color"
)
var rateLimiter = time.NewTicker(500 * time.Millisecond)
func main() {
defer arn.Node.Close()
// Replace this with ID list from twist.moe later
twistAnime, err := twist.GetAnimeIndex()
arn.PanicOnError(err)
idList := arn.IDList(twistAnime.KitsuIDs())
// Save index in cache
arn.DB.Set("IDList", "animetwist index", &idList)
color.Yellow("Refreshing twist.moe links for %d anime", len(idList))
for count, animeID := range idList {
anime, animeErr := arn.GetAnime(animeID)
if animeErr != nil {
color.Red("Error fetching anime from the database with ID %s: %v", animeID, animeErr)
continue
}
// Log
fmt.Fprintf(os.Stdout, "[%d / %d] ", count+1, len(idList))
// Refresh
anime.RefreshEpisodes()
// Ok
color.Green("Found %d episodes for anime %s", len(anime.Episodes().Items), animeID)
// Wait for rate limiter
<-rateLimiter.C
}
}

4
layout/layout.go
View File

@ -2,6 +2,7 @@ package layout
import (
"github.com/aerogo/aero"
"github.com/animenotifier/arn"
"github.com/animenotifier/notify.moe/components"
"github.com/animenotifier/notify.moe/utils"
)
@ -9,5 +10,6 @@ import (
// Render layout.
func Render(ctx *aero.Context, content string) string {
user := utils.GetUser(ctx)
return components.Layout(ctx.App, ctx, user, content)
openGraph, _ := ctx.Data.(*arn.OpenGraph)
return components.Layout(ctx.App, ctx, user, openGraph, content)
}

45
layout/layout.pixy
View File

@ -1,29 +1,36 @@
component Layout(app *aero.Application, ctx *aero.Context, user *arn.User, content string)
component Layout(app *aero.Application, ctx *aero.Context, user *arn.User, openGraph *arn.OpenGraph, content string)
html(lang="en")
head
title= app.Config.Title
if openGraph != nil
title= openGraph.Tags["og:title"]
else
title= app.Config.Title
meta(name="viewport", content="width=device-width, minimum-scale=1.0, initial-scale=1.0, user-scalable=yes")
meta(name="theme-color", content=app.Config.Manifest.ThemeColor)
if openGraph != nil
for name, value := range openGraph.Meta
meta(name=name, content=value)
for property, content := range openGraph.Tags
meta(property=property, content=content)
link(rel="chrome-webstore-item", href="https://chrome.google.com/webstore/detail/hajchfikckiofgilinkpifobdbiajfch")
link(rel="manifest", href="/manifest.json")
body
#container(class=utils.GetContainerClass(ctx))
#header
Navigation(user)
#content-container
main#content.fade!= content
LoadingAnimation
//- #header
//- Navigation(user)
#columns
Sidebar(user)
Content(content)
LoadingAnimation
StatusMessage
if user != nil
#user(data-id=user.ID)
script(src="/scripts")
component LoadingAnimation
#loading.sk-cube-grid.fade
.sk-cube.hide
.sk-cube
.sk-cube.hide
.sk-cube
.sk-cube.sk-cube-center
.sk-cube
.sk-cube.hide
.sk-cube
.sk-cube.hide
component Content(content string)
#content-container
main#content.fade!= content

66
layout/sidebar/sidebar.pixy Normal file
View File

@ -0,0 +1,66 @@
component Sidebar(user *arn.User)
aside#sidebar
.user-image-container
if user != nil
Avatar(user)
else
img.user-image.lazy(src=utils.EmptyImage(), data-src="/images/brand/64.png", data-webp="true", alt="Anime Notifier")
if user != nil
SidebarButton("Home", "/animelist/watching", "home")
//- SidebarButton("Dash", "/dashboard", "tachometer")
else
SidebarButton("Home", "/", "home")
SidebarButton("Forum", "/forum", "comment")
SidebarButton("Explore", "/explore", "th")
//- SidebarButton("Artworks", "/artworks", "paint-brush")
SidebarButton("Soundtracks", "/soundtracks", "headphones")
//- SidebarButton("AMVs", "/amvs", "video-camera")
//- SidebarButton("Games", "/games", "gamepad")
SidebarButton("Users", "/users", "globe")
//- SidebarButton("Search", "/search", "search")
if user != nil
//- if user.Role == "admin"
//- SidebarButton("Groups", "/groups", "users")
SidebarButton("Shop", "/shop", "shopping-cart")
//- if user.Role == "admin" || user.Role == "editor"
//- SidebarButton("Statistics", "/statistics", "pie-chart")
SidebarButton("Settings", "/settings", "cog")
.spacer
.sidebar-link(aria-label="Search")
.sidebar-button
Icon("search")
FuzzySearch
if user != nil
if user.Role == "admin"
SidebarButton("Admin", "/admin", "wrench")
if user.Role == "editor"
SidebarButton("Editor", "/editor", "pencil")
SidebarButton("Help", "/thread/I3MMiOtzR", "question-circle")
if user != nil
SidebarButtonNoAJAX("Logout", "/logout", "sign-out")
else
SidebarButton("Login", "/login", "sign-in")
component SidebarButton(name string, target string, icon string)
a.sidebar-link.ajax(href=target, aria-label=name, data-bubble="true")
.sidebar-button
Icon(icon)
span.sidebar-text= name
component SidebarButtonNoAJAX(name string, target string, icon string)
a.sidebar-link(href=target, aria-label=name, data-bubble="true")
.sidebar-button
Icon(icon)
span.sidebar-text= name

60
layout/sidebar/sidebar.scarlet Normal file
View File

@ -0,0 +1,60 @@
sidebar-spacing-y = 0.7rem
#sidebar
vertical
position fixed
left 0
top 0
z-index 10
min-width 200px
height 100%
background sidebar-opaque-background
transform translateX(-100%)
overflow-x hidden
overflow-y auto
opacity 0
pointer-events none
box-shadow shadow-medium
transition opacity transition-speed ease, transform transition-speed ease
will-change opacity, transition
.user-image-container
horizontal
justify-content center
margin 0.8rem 0
flex-shrink 0
> 800px
#sidebar
opacity 1
transform none
position static
pointer-events auto
box-shadow none
border-right ui-border
background sidebar-background
.sidebar-visible
transform translateX(0) !important
pointer-events auto !important
opacity 1 !important
.sidebar-link
color text-color
&.active
.sidebar-button
color tab-active-color
background tab-active-background
text-shadow tab-active-text-shadow
background tab-active-background
.sidebar-button
horizontal
align-items center
padding sidebar-spacing-y 1rem
// background ui-background
.icon
font-size 1rem
margin-right 0.75rem

193
main.go
View File

@ -2,31 +2,56 @@ package main
import (
"github.com/aerogo/aero"
"github.com/aerogo/api"
"github.com/aerogo/session-store-nano"
"github.com/animenotifier/arn"
"github.com/animenotifier/notify.moe/auth"
"github.com/animenotifier/notify.moe/components"
"github.com/animenotifier/notify.moe/components/css"
"github.com/animenotifier/notify.moe/layout"
"github.com/animenotifier/notify.moe/middleware"
"github.com/animenotifier/notify.moe/pages/admin"
"github.com/animenotifier/notify.moe/pages/airing"
"github.com/animenotifier/notify.moe/pages/amvs"
"github.com/animenotifier/notify.moe/pages/anime"
"github.com/animenotifier/notify.moe/pages/animelist"
"github.com/animenotifier/notify.moe/pages/animelistitem"
"github.com/animenotifier/notify.moe/pages/apiview"
"github.com/animenotifier/notify.moe/pages/artworks"
"github.com/animenotifier/notify.moe/pages/best"
"github.com/animenotifier/notify.moe/pages/character"
"github.com/animenotifier/notify.moe/pages/charge"
"github.com/animenotifier/notify.moe/pages/compare"
"github.com/animenotifier/notify.moe/pages/dashboard"
"github.com/animenotifier/notify.moe/pages/database"
"github.com/animenotifier/notify.moe/pages/editanime"
"github.com/animenotifier/notify.moe/pages/editor"
"github.com/animenotifier/notify.moe/pages/embed"
"github.com/animenotifier/notify.moe/pages/explore"
"github.com/animenotifier/notify.moe/pages/forum"
"github.com/animenotifier/notify.moe/pages/forums"
"github.com/animenotifier/notify.moe/pages/group"
"github.com/animenotifier/notify.moe/pages/groups"
"github.com/animenotifier/notify.moe/pages/home"
"github.com/animenotifier/notify.moe/pages/inventory"
"github.com/animenotifier/notify.moe/pages/listimport"
"github.com/animenotifier/notify.moe/pages/listimport/listimportanilist"
"github.com/animenotifier/notify.moe/pages/listimport/listimportkitsu"
"github.com/animenotifier/notify.moe/pages/listimport/listimportmyanimelist"
"github.com/animenotifier/notify.moe/pages/login"
"github.com/animenotifier/notify.moe/pages/popularanime"
"github.com/animenotifier/notify.moe/pages/me"
"github.com/animenotifier/notify.moe/pages/newthread"
"github.com/animenotifier/notify.moe/pages/notifications"
"github.com/animenotifier/notify.moe/pages/paypal"
"github.com/animenotifier/notify.moe/pages/posts"
"github.com/animenotifier/notify.moe/pages/profile"
"github.com/animenotifier/notify.moe/pages/search"
"github.com/animenotifier/notify.moe/pages/settings"
"github.com/animenotifier/notify.moe/pages/shop"
"github.com/animenotifier/notify.moe/pages/soundtrack"
"github.com/animenotifier/notify.moe/pages/soundtracks"
"github.com/animenotifier/notify.moe/pages/statistics"
"github.com/animenotifier/notify.moe/pages/threads"
"github.com/animenotifier/notify.moe/pages/user"
"github.com/animenotifier/notify.moe/pages/users"
"github.com/animenotifier/notify.moe/pages/webdev"
)
var app = aero.New()
@ -44,45 +69,154 @@ func configure(app *aero.Application) *aero.Application {
app.SetStyle(css.Bundle())
// Sessions
app.Sessions.Duration = 3600 * 24
app.Sessions.Store = arn.NewAerospikeStore("Session", app.Sessions.Duration)
app.Sessions.Duration = 3600 * 24 * 30 * 6
// TODO: ...
// app.Sessions.Store = aerospikestore.New(arn.DB, "Session", app.Sessions.Duration)
app.Sessions.Store = nanostore.New(arn.DB.Collection("Session"))
// Layout
app.Layout = layout.Render
// Ajax routes
app.Ajax("/", dashboard.Get)
app.Ajax("/anime", popularanime.Get)
app.Ajax("/", home.Get)
app.Ajax("/dashboard", dashboard.Get)
app.Ajax("/anime/:id", anime.Get)
app.Ajax("/anime/:id/episodes", anime.Episodes)
app.Ajax("/anime/:id/characters", anime.Characters)
app.Ajax("/anime/:id/tracks", anime.Tracks)
app.Ajax("/anime/:id/edit", editanime.Get)
app.Ajax("/api", apiview.Get)
app.Ajax("/best/anime", best.Get)
app.Ajax("/explore", explore.Get)
app.Ajax("/forum", forums.Get)
app.Ajax("/forum/:tag", forum.Get)
app.Ajax("/threads/:id", threads.Get)
app.Ajax("/posts/:id", posts.Get)
app.Ajax("/thread/:id", threads.Get)
app.Ajax("/post/:id", posts.Get)
app.Ajax("/character/:id", character.Get)
app.Ajax("/new/thread", newthread.Get)
app.Ajax("/artworks", artworks.Get)
app.Ajax("/amvs", amvs.Get)
app.Ajax("/users", users.Active)
app.Ajax("/users/osu", users.Osu)
app.Ajax("/users/staff", users.Staff)
app.Ajax("/statistics", statistics.Get)
app.Ajax("/statistics/anime", statistics.Anime)
app.Ajax("/login", login.Get)
// Settings
app.Ajax("/settings", settings.Get(components.SettingsPersonal))
app.Ajax("/settings/accounts", settings.Get(components.SettingsAccounts))
app.Ajax("/settings/notifications", settings.Get(components.SettingsNotifications))
app.Ajax("/settings/apps", settings.Get(components.SettingsApps))
app.Ajax("/settings/avatar", settings.Get(components.SettingsAvatar))
app.Ajax("/settings/formatting", settings.Get(components.SettingsFormatting))
app.Ajax("/settings/pro", settings.Get(components.SettingsPro))
// Soundtracks
app.Ajax("/soundtracks", soundtracks.Get)
app.Ajax("/soundtracks/from/:index", soundtracks.From)
app.Ajax("/soundtrack/:id", soundtrack.Get)
app.Ajax("/soundtrack/:id/edit", soundtrack.Edit)
// Groups
app.Ajax("/groups", groups.Get)
app.Ajax("/group/:id", group.Get)
app.Ajax("/group/:id/edit", group.Edit)
app.Ajax("/group/:id/forum", group.Forum)
// User profiles
app.Ajax("/user", user.Get)
app.Ajax("/user/:nick", profile.Get)
app.Ajax("/user/:nick/threads", profile.GetThreadsByUser)
app.Ajax("/user/:nick/posts", profile.GetPostsByUser)
app.Ajax("/user/:nick/soundtracks", profile.GetSoundTracksByUser)
app.Ajax("/user/:nick/stats", profile.GetStatsByUser)
app.Ajax("/user/:nick/followers", profile.GetFollowers)
app.Ajax("/user/:nick/animelist", animelist.Get)
app.Ajax("/user/:nick/animelist/:id", animelistitem.Get)
app.Ajax("/settings", settings.Get)
app.Ajax("/admin", admin.Get)
app.Ajax("/user/:nick/animelist/watching", animelist.FilterByStatus(arn.AnimeListStatusWatching))
app.Ajax("/user/:nick/animelist/completed", animelist.FilterByStatus(arn.AnimeListStatusCompleted))
app.Ajax("/user/:nick/animelist/planned", animelist.FilterByStatus(arn.AnimeListStatusPlanned))
app.Ajax("/user/:nick/animelist/hold", animelist.FilterByStatus(arn.AnimeListStatusHold))
app.Ajax("/user/:nick/animelist/dropped", animelist.FilterByStatus(arn.AnimeListStatusDropped))
app.Ajax("/user/:nick/animelist/anime/:id", animelistitem.Get)
// Anime list
app.Ajax("/animelist/watching", home.FilterByStatus(arn.AnimeListStatusWatching))
app.Ajax("/animelist/completed", home.FilterByStatus(arn.AnimeListStatusCompleted))
app.Ajax("/animelist/planned", home.FilterByStatus(arn.AnimeListStatusPlanned))
app.Ajax("/animelist/hold", home.FilterByStatus(arn.AnimeListStatusHold))
app.Ajax("/animelist/dropped", home.FilterByStatus(arn.AnimeListStatusDropped))
// Compare
app.Ajax("/compare/animelist/:nick-1/:nick-2", compare.AnimeList)
// Search
app.Ajax("/search", search.Get)
app.Ajax("/search/:term", search.Get)
app.Ajax("/users", users.Get)
app.Ajax("/login", login.Get)
app.Ajax("/airing", airing.Get)
app.Ajax("/webdev", webdev.Get)
app.Ajax("/extension/embed", embed.Get)
// Shop
app.Ajax("/shop", shop.Get)
app.Ajax("/inventory", inventory.Get)
app.Ajax("/charge", charge.Get)
app.Ajax("/shop/history", shop.PurchaseHistory)
app.Post("/api/shop/buy/:item/:quantity", shop.BuyItem)
// Admin
app.Ajax("/admin", admin.Get)
app.Ajax("/admin/webdev", admin.WebDev)
app.Ajax("/admin/purchases", admin.PurchaseHistory)
// Editor
app.Ajax("/editor", editor.Get)
app.Ajax("/editor/anilist", editor.AniList)
app.Ajax("/editor/shoboi", editor.Shoboi)
// Mixed
app.Ajax("/database", database.Get)
app.Get("/api/select/:data-type/where/:field/is/:field-value", database.Select)
// Import
app.Ajax("/import", listimport.Get)
app.Ajax("/import/anilist/animelist", listimportanilist.Preview)
app.Ajax("/import/anilist/animelist/finish", listimportanilist.Finish)
app.Ajax("/import/myanimelist/animelist", listimportmyanimelist.Preview)
app.Ajax("/import/myanimelist/animelist/finish", listimportmyanimelist.Finish)
app.Ajax("/import/kitsu/animelist", listimportkitsu.Preview)
app.Ajax("/import/kitsu/animelist/finish", listimportkitsu.Finish)
// Genres
// app.Ajax("/genres", genres.Get)
// app.Ajax("/genres/:name", genre.Get)
// Middleware
app.Use(middleware.Log())
app.Use(middleware.Session())
app.Use(middleware.UserInfo())
// Browser extension
app.Ajax("/extension/embed", embed.Get)
// API
api := api.New("/api/", arn.DB)
api.Install(app)
app.Get("/api/me", me.Get)
app.Get("/api/test/notification", notifications.Test)
// PayPal
app.Ajax("/paypal/success", paypal.Success)
app.Ajax("/paypal/cancel", paypal.Cancel)
app.Post("/api/paypal/payment/create", paypal.CreatePayment)
// Assets
configureAssets(app)
// Rewrite
app.Rewrite(rewrite)
// Middleware
app.Use(
middleware.Firewall(),
middleware.Log(),
middleware.Session(),
middleware.UserInfo(),
)
// API
arn.API.Install(app)
// Domain
if arn.IsDevelopment() {
@ -92,5 +226,16 @@ func configure(app *aero.Application) *aero.Application {
// Authentication
auth.Install(app)
// Close the database node on shutdown
app.OnShutdown(arn.Node.Close)
// Prefetch data from all collections
arn.DB.PrefetchData()
// Specify test routes
for route, examples := range routeTests {
app.Test(route, examples)
}
return app
}

5
main_test.go
View File

@ -1,6 +1,7 @@
package main
import (
"fmt"
"net/http"
"net/http/httptest"
"testing"
@ -11,7 +12,7 @@ import (
func TestRoutes(t *testing.T) {
app := configure(aero.New())
for _, examples := range tests {
for _, examples := range routeTests {
for _, example := range examples {
request, err := http.NewRequest("GET", example, nil)
@ -23,7 +24,7 @@ func TestRoutes(t *testing.T) {
app.Handler().ServeHTTP(responseRecorder, request)
if status := responseRecorder.Code; status != http.StatusOK {
t.Errorf("%s | Wrong status code | %v instead of %v", example, status, http.StatusOK)
panic(fmt.Errorf("%s | Wrong status code | %v instead of %v", example, status, http.StatusOK))
}
}
}

21
makefile
View File

@ -3,27 +3,40 @@
GOCMD=@go
GOBUILD=$(GOCMD) build
GOINSTALL=$(GOCMD) install
GOTEST=$(GOCMD) test
GOTEST=@./go-test-color.sh
BUILDJOBS=@./jobs/build.sh
BUILDPATCHES=@./patches/build.sh
BUILDBOTS=@./bots/build.sh
TSCMD=@tsc
IPTABLES=@sudo iptables
server:
$(GOBUILD)
jobs:
$(BUILDJOBS)
bots:
$(BUILDBOTS)
patches:
$(BUILDPATCHES)
js:
$(TSCMD)
install:
$(GOINSTALL)
test:
$(GOTEST)
$(GOTEST) github.com/animenotifier/... -v -cover
bench:
$(GOTEST) -bench .
tools:
go get -u golang.org/x/tools/cmd/goimports
go get -u github.com/aerogo/pack
go get -u github.com/aerogo/run
go install github.com/aerogo/pack
go install github.com/aerogo/run
versions:
@go version
@asd --version
assets:
$(TSCMD)
@pack
depslist:
$(GOCMD) list -f {{.Deps}} | sed -e 's/\[//g' -e 's/\]//g' | tr " " "\n"
@ -32,6 +45,6 @@ clean:
ports:
$(IPTABLES) -t nat -A OUTPUT -o lo -p tcp --dport 80 -j REDIRECT --to-port 4000
$(IPTABLES) -t nat -A OUTPUT -o lo -p tcp --dport 443 -j REDIRECT --to-port 4001
all: assets server jobs patches
all: assets server bots jobs patches
.PHONY: jobs patches ports
.PHONY: bots jobs patches ports

79
middleware/Firewall.go Normal file
View File

@ -0,0 +1,79 @@
package middleware
import (
"strings"
"time"
"github.com/aerogo/aero"
"github.com/animenotifier/notify.moe/utils"
cache "github.com/patrickmn/go-cache"
)
const requestThreshold = 10
var ipToStats = cache.New(15*time.Minute, 15*time.Minute)
// IPStats captures the statistics for a single IP.
type IPStats struct {
Requests []string
}
// Firewall middleware detects malicious requests.
func Firewall() aero.Middleware {
return func(ctx *aero.Context, next func()) {
var stats *IPStats
ip := ctx.RealIP()
// Allow localhost
if ip == "127.0.0.1" {
next()
return
}
statsObj, found := ipToStats.Get(ip)
if found {
stats = statsObj.(*IPStats)
} else {
stats = &IPStats{
Requests: []string{},
}
ipToStats.Set(ip, stats, cache.DefaultExpiration)
}
// Add requested URI to the list of requests
stats.Requests = append(stats.Requests, ctx.URI())
if len(stats.Requests) > requestThreshold {
stats.Requests = stats.Requests[len(stats.Requests)-requestThreshold:]
for _, uri := range stats.Requests {
// Allow request
if strings.Contains(uri, "/_/") || strings.Contains(uri, "/api/") || strings.Contains(uri, "/scripts") || strings.Contains(uri, "/service-worker") || strings.Contains(uri, "/favicon.ico") || strings.Contains(uri, "/extension/embed") {
next()
return
}
}
// Allow logged in users
if ctx.HasSession() {
user := utils.GetUser(ctx)
if user != nil {
// Allow request
next()
return
}
}
// Disallow request
request.Error("[guest]", ip, "BLOCKED BY FIREWALL", ctx.URI())
return
}
// Allow the request if the number of requests done by the IP is below the threshold
next()
}
}

28
middleware/UserInfo.go
View File

@ -1,31 +1,18 @@
package middleware
import (
"encoding/json"
"io/ioutil"
"net/http"
"strconv"
"strings"
"github.com/aerogo/aero"
"github.com/aerogo/http/client"
"github.com/animenotifier/arn"
"github.com/animenotifier/notify.moe/utils"
"github.com/fatih/color"
"github.com/mssola/user_agent"
"github.com/parnurzeal/gorequest"
)
var apiKeys arn.APIKeys
func init() {
data, _ := ioutil.ReadFile("security/api-keys.json")
err := json.Unmarshal(data, &apiKeys)
if err != nil {
panic(err)
}
}
// UserInfo updates user related information after each request.
func UserInfo() aero.Middleware {
return func(ctx *aero.Context, next func()) {
@ -84,22 +71,21 @@ func updateUserInfo(ctx *aero.Context, user *arn.User) {
// Updates the location of the user.
func updateUserLocation(user *arn.User, newIP string) {
user.IP = newIP
locationAPI := "https://api.ipinfodb.com/v3/ip-city/?key=" + apiKeys.IPInfoDB.ID + "&ip=" + user.IP + "&format=json"
locationAPI := "https://api.ipinfodb.com/v3/ip-city/?key=" + arn.APIKeys.IPInfoDB.ID + "&ip=" + user.IP + "&format=json"
response, err := client.Get(locationAPI).End()
response, data, err := gorequest.New().Get(locationAPI).EndBytes()
if len(err) > 0 && err[0] != nil {
color.Red("Couldn't fetch location data | Error: %s | IP: %s", err[0].Error(), user.IP)
if err != nil {
color.Red("Couldn't fetch location data | Error: %s | IP: %s", err.Error(), user.IP)
return
}
if response.StatusCode != http.StatusOK {
if response.StatusCode() != http.StatusOK {
color.Red("Couldn't fetch location data | Status: %d | IP: %s", response.StatusCode, user.IP)
return
}
newLocation := arn.IPInfoDBLocation{}
json.Unmarshal(data, &newLocation)
response.Unmarshal(&newLocation)
if newLocation.CountryName != "-" {
user.Location.CountryName = newLocation.CountryName

2
mixins/AnimeGrid.pixy
View File

@ -2,4 +2,4 @@ component AnimeGrid(animeList []*arn.Anime)
.anime-grid
each anime in animeList
a.anime-grid-cell.ajax(href="/anime/" + toString(anime.ID))
img.anime-grid-image.lazy(data-src=anime.Image.Small, alt=anime.Title.Romaji, title=anime.Title.Romaji + " (" + toString(anime.Rating.Overall) + ")")
img.anime-grid-image.lazy(data-src=anime.Image.Small, alt=anime.Title.Romaji, title=anime.Title.Romaji)

7
mixins/Avatar.pixy
View File

@ -1,10 +1,13 @@
component Avatar(user *arn.User)
a.user.ajax(href="/+" + user.Nick, title=user.Nick)
CustomAvatar(user, user.Link(), user.Nick)
component CustomAvatar(user *arn.User, link string, title string)
a.user.ajax(href=link, title=title)
AvatarNoLink(user)
component AvatarNoLink(user *arn.User)
if user.HasAvatar()
img.user-image.lazy(data-src=user.SmallAvatar(), alt=user.Nick)
img.user-image.lazy(data-src=user.SmallAvatar(), data-webp="true", alt=user.Nick)
else
SVGAvatar

4
mixins/Character.pixy Normal file
View File

@ -0,0 +1,4 @@
component Character(character *arn.Character)
a.character.ajax(href="/character/" + character.ID)
img.character-image.lazy(data-src=character.Image, alt=character.Name, title=character.Name)
//- span.character-name= character.Name

6
mixins/ForumTags.pixy
View File

@ -1,5 +1,5 @@
component ForumTags
.buttons.forum-tags
.tabs
ForumTag("All", "", "list")
ForumTag("General", "general", "list")
ForumTag("News", "news", "list")
@ -9,6 +9,6 @@ component ForumTags
ForumTag("Bugs", "bug", "list")
component ForumTag(title string, category string, icon string)
a.button.forum-tag.action(href=strings.TrimSuffix("/forum/" + category, "/"), data-action="diff", data-trigger="click")
a.tab.action(href=strings.TrimSuffix("/forum/" + category, "/"), data-action="diff", data-trigger="click")
Icon(arn.GetForumIcon(category))
span.forum-tag-text= title
span.tab-text= title

2
mixins/FuzzySearch.pixy Normal file
View File

@ -0,0 +1,2 @@
component FuzzySearch
input#search.action(data-action="search", data-trigger="input", type="text", placeholder="Search...", title="Shortcut: F")

36
mixins/Input.pixy
View File

@ -1,19 +1,37 @@
component InputText(id string, value string, label string, placeholder string)
.widget-input
.widget-section
label(for=id)= label + ":"
input.widget-element.action(id=id, type="text", value=value, placeholder=placeholder, title=placeholder, data-action="save", data-trigger="change")
input.widget-ui-element.action(id=id, data-field=id, type="text", value=value, placeholder=placeholder, title=placeholder, data-action="save", data-trigger="change")
component InputTextArea(id string, value string, label string, placeholder string)
.widget-input
.widget-section
label(for=id)= label + ":"
textarea.widget-element.action(id=id, placeholder=placeholder, title=placeholder, data-action="save", data-trigger="change")= value
textarea.widget-ui-element.action(id=id, data-field=id, placeholder=placeholder, title=placeholder, data-action="save", data-trigger="change")= value
component InputNumber(id string, value float64, label string, placeholder string, min string, max string, step string)
.widget-input
.widget-section
label(for=id)= label + ":"
input.widget-element.action(id=id, type="number", value=value, min=min, max=max, step=step, placeholder=placeholder, title=placeholder, data-action="save", data-trigger="change")
input.widget-ui-element.action(id=id, data-field=id, type="number", value=value, min=min, max=max, step=step, placeholder=placeholder, title=placeholder, data-action="save", data-trigger="change")
component InputSelection(id string, value string, label string, placeholder string)
.widget-input
component InputSelection(id string, value string, label string, placeholder string, options []*arn.Option)
.widget-section
label(for=id)= label + ":"
select.widget-element.action(id=id, value=value, title=placeholder, data-action="save", data-trigger="change")
select.widget-ui-element.action(id=id, data-field=id, value=value, title=placeholder, data-action="save", data-trigger="change")
each option in options
option(value=option.Value)= option.Label
component InputTags(id string, value []string, label string, tooltip string)
.widget-section
label(for=id)= label + ":"
.tags(id=id)
for index, tag := range value
.tag.tag-edit
span.tag-title.action(contenteditable="true", data-trigger="focusout", data-action="save", data-field=id + "[" + strconv.Itoa(index) + "]")= tag
button.tag-remove.action(data-action="arrayRemove", data-trigger="click", data-field=id, data-index=index)
RawIcon("trash")
button.tag-add.action(data-action="arrayAppend", data-trigger="click", data-field=id)
RawIcon("plus")
p!= tooltip

12
mixins/Japanese.pixy Normal file
View File

@ -0,0 +1,12 @@
component Japanese(text string)
if arn.ContainsUnicodeLetters(text)
for _, token := range arn.JapaneseTokenizer.Tokenize(text)
if token.Furigana
a.japanese(href="http://jisho.org/search/" + token.Original, target="_blank", rel="noopener")
ruby(title=token.Romaji)= token.Original
rt.furigana= token.Hiragana
else
ruby.japanese(title=token.Romaji)= token.Original
rt.furigana
else
span.japanese= text

4
mixins/LoadMore.pixy Normal file
View File

@ -0,0 +1,4 @@
component LoadMore(index int)
button#load-more-button.action(data-action="loadMore", data-trigger="click", data-index=index)
Icon("refresh")
span Load more

11
mixins/LoadingAnimation.pixy Normal file
View File

@ -0,0 +1,11 @@
component LoadingAnimation
#loading.sk-cube-grid.fade
.sk-cube.hide
.sk-cube
.sk-cube.hide
.sk-cube
.sk-cube.sk-cube-center
.sk-cube
.sk-cube.hide
.sk-cube
.sk-cube.hide

57
mixins/Navigation.pixy
View File

@ -1,57 +0,0 @@
component Navigation(user *arn.User)
if user == nil
LoggedOutMenu
else
LoggedInMenu(user)
component LoggedOutMenu
nav#navigation.logged-out
NavigationButton("About", "/", "question-circle")
NavigationButton("Anime", "/anime", "television")
NavigationButton("Forum", "/forum", "comment")
FuzzySearch
.extra-navigation
NavigationButton("Users", "/users", "globe")
NavigationButton("Airing", "/airing", "th")
NavigationButton("Login", "/login", "sign-in")
component LoggedInMenu(user *arn.User)
nav#navigation.logged-in
.extension-navigation
NavigationButton("Watching list", "/extension/embed", "home")
NavigationButton("Dash", "/", "dashboard")
NavigationButton("Profile", "/+", "user")
NavigationButton("Forum", "/forum", "comment")
NavigationButton("Anime", "/anime", "television")
FuzzySearch
.extra-navigation
NavigationButton("Users", "/users", "globe")
.extra-navigation
NavigationButton("Airing", "/airing", "th")
NavigationButton("Settings", "/settings", "cog")
.extra-navigation
NavigationButtonNoAJAX("Logout", "/logout", "sign-out")
component FuzzySearch
input#search.action(data-action="search", data-trigger="input", type="text", placeholder="Search...", title="Shortcut: F")
component NavigationButton(name string, target string, icon string)
a.navigation-link.ajax(href=target, aria-label=name, title=name)
.navigation-button
Icon(icon)
span.navigation-text= name
component NavigationButtonNoAJAX(name string, target string, icon string)
a.navigation-link(href=target, aria-label=name)
.navigation-button
Icon(icon)
span.navigation-text= name

48
mixins/Postable.pixy
View File

@ -1,5 +1,5 @@
component Postable(post arn.Postable, highlightAuthorID string)
.post.mountable(data-highlight=post.Author().ID == highlightAuthorID)
component Postable(post arn.Postable, user *arn.User, highlightAuthorID string)
.post.mountable(id=strings.ToLower(post.Type()) + "-" + toString(post.ID()), data-highlight=post.Author().ID == highlightAuthorID, data-pro=post.Author().IsPro(), data-api="/api/" + strings.ToLower(post.Type()) + "/" + post.ID())
.post-author
Avatar(post.Author())
@ -9,30 +9,38 @@ component Postable(post arn.Postable, highlightAuthorID string)
.post-content
div(id="render-" + post.ID())!= post.HTML()
//- if user && user.ID === post.authorId
//- textarea.post-input.hidden(id="source-" + post.ID)= post.text
//- a.post-save.hidden(id="save-" + post.ID, onclick=`$.saveEdit("${type.toLowerCase()}", "${post.ID}")`)
//- i.fa.fa-save
//- span Save
if user != nil && user.ID == post.Author().ID
.post-edit-interface
if post.Type() == "Thread"
input.post-title-input.hidden(id="title-" + post.ID(), value=post.Title(), type="text", placeholder="Thread title")
textarea.post-text-input.hidden(id="source-" + post.ID())= post.Text()
.buttons.hidden(id="edit-toolbar-" + post.ID())
a.button.post-save.action(data-action="savePost", data-trigger="click", data-id=post.ID())
Icon("save")
span Save
a.button.post-cancel-edit.action(data-action="editPost", data-trigger="click", data-id=post.ID())
Icon("close")
span Cancel
.post-date.utc-date(data-date=post.Created())
.post-toolbar(id="toolbar-" + post.ID())
.spacer
.post-likes(id="likes-" + post.ID(), title="Likes")= len(post.Likes())
//- if user != nil
//- if user.ID !== post.authorId
//- - var liked = post.likes && post.likes.indexOf(user.ID) !== -1
if user != nil
if user.ID != post.Author().ID
if post.LikedBy(user.ID)
a.post-tool.post-unlike.action(id="unlike-" + post.ID(), title="Unlike", data-action="unlike", data-trigger="click")
Icon("thumbs-down")
else
a.post-tool.post-like.action(id="like-" + post.ID(), title="Like", data-action="like", data-trigger="click")
Icon("thumbs-up")
//- a.post-tool.post-like(id="like-" + post.ID, onclick=`$.like("${type.toLowerCase()}", "${post.ID}")`, title="Like", class=liked ? "hidden" : ")
//- i.fa.fa-thumbs-up.fa-fw
//- a.post-tool.post-unlike(id="unlike-" + post.ID, onclick=`$.unlike("${type.toLowerCase()}", "${post.ID}")`, title="Unlike", class=!liked ? "hidden" : ")
//- i.fa.fa-thumbs-down.fa-fw
//- if type === "Posts" || type === "Threads"
//- if user.ID === post.authorId
//- a.post-tool.post-edit(onclick=`$.edit("${post.ID}")`, title="Edit")
//- i.fa.fa-pencil.fa-fw
if user.ID == post.Author().ID
a.post-tool.post-edit.action(data-action="editPost", data-trigger="click", data-id=post.ID(), title="Edit")
Icon("pencil")
if post.Type() != "Thread"
a.post-tool.post-permalink.ajax(href=post.Link(), title="Permalink")

4
mixins/PostableList.pixy
View File

@ -1,5 +1,5 @@
component PostableList(postables []arn.Postable)
component PostableList(postables []arn.Postable, user *arn.User)
.thread
.posts
each post in postables
Postable(post, "")
Postable(post, user, "")

2
mixins/ProfileImage.pixy
View File

@ -1,6 +1,6 @@
component ProfileImage(user *arn.User)
if user.HasAvatar()
img.profile-image(src=user.LargeAvatar(), alt="Profile image")
img.profile-image.lazy(data-src=user.LargeAvatar(), data-webp="true", alt="Profile image")
else
svg.profile-image(viewBox="0 0 50 50", alt="Profile image")
circle.head(cx="25", cy="19", r="10")

7
mixins/Rating.pixy
View File

@ -1,2 +1,5 @@
component Rating(value float64)
.anime-rating= int(value / 10 + 0.5)
component Rating(value float64, user *arn.User)
if user == nil
.anime-rating= fmt.Sprintf("%.1f", value)
else
.anime-rating= fmt.Sprintf("%." + strconv.Itoa(user.Settings().Format.RatingsPrecision) + "f", value)

29
mixins/SoundTrack.pixy Normal file
View File

@ -0,0 +1,29 @@
component SoundTrack(track *arn.SoundTrack)
SoundTrackMedia(track, track.Media[0])
component SoundTrackMedia(track *arn.SoundTrack, media *arn.ExternalMedia)
.sound-track.mountable(id=track.ID)
SoundTrackContent(track, media)
SoundTrackFooter(track)
component SoundTrackContent(track *arn.SoundTrack, media *arn.ExternalMedia)
.sound-track-content
if track.MainAnime() != nil
a.sound-track-anime-link.ajax(href="/anime/" + track.MainAnime().ID)
img.sound-track-anime-image.lazy(data-src=track.MainAnime().Image.Small, alt=track.MainAnime().Title.Canonical, title=track.MainAnime().Title.Canonical)
ExternalMedia(media)
component SoundTrackFooter(track *arn.SoundTrack)
.sound-track-footer
if track.Title == ""
a.ajax(href=track.Link() + "/edit") untitled
else
a.ajax(href=track.Link())= track.Title
span posted
span.utc-date(data-date=track.Created)
span by
a.ajax(href=track.Creator().Link())= track.Creator().Nick + " "
component ExternalMedia(media *arn.ExternalMedia)
iframe.lazy(data-src=media.EmbedLink(), allowfullscreen="allowfullscreen")

5
mixins/StatusMessage.pixy Normal file
View File

@ -0,0 +1,5 @@
component StatusMessage
#status-message.fade.fade-out
#status-message-text
a.status-message-action.action(href="#", data-trigger="click", data-action="closeStatusMessage", aria-label="Close status message")
RawIcon("close")

7
mixins/StatusTabs.pixy Normal file
View File

@ -0,0 +1,7 @@
component StatusTabs(urlPrefix string)
.tabs
Tab("Watching", "play", urlPrefix + "/watching")
Tab("Completed", "check", urlPrefix + "/completed")
Tab("Planned", "forward", urlPrefix + "/planned")
Tab("On Hold", "pause", urlPrefix + "/hold")
Tab("Dropped", "stop", urlPrefix + "/dropped")

4
mixins/Tab.pixy Normal file
View File

@ -0,0 +1,4 @@
component Tab(label string, icon string, url string)
a.tab.action(href=url, data-action="diff", data-trigger="click", aria-label=label)
Icon(icon)
span.tab-text= label

6
mixins/ThreadLink.pixy
View File

@ -4,10 +4,10 @@ component ThreadLink(thread *arn.Thread)
Avatar(thread.Author())
.thread-content-container
.thread-content
if thread.Sticky
if thread.Sticky != 0
Icon("thumb-tack")
a.thread-link-title.ajax(href="/threads/" + thread.ID)= thread.Title
a.thread-link-title.ajax(href="/thread/" + thread.ID)= thread.Title
.spacer
.thread-reply-count= thread.Replies
.thread-reply-count= len(thread.Posts)
.thread-icons
Icon(arn.GetForumIcon(thread.Tags[0]))

50
pages/admin/admin.go
View File

@ -1,29 +1,61 @@
package admin
import (
"sort"
"github.com/aerogo/aero"
"github.com/animenotifier/arn"
"github.com/animenotifier/notify.moe/components"
"github.com/animenotifier/notify.moe/utils"
"github.com/shirou/gopsutil/host"
)
// Get admin page.
func Get(ctx *aero.Context) string {
user := utils.GetUser(ctx)
if user == nil || user.Role != "admin" {
if user == nil || (user.Role != "admin" && user.Role != "editor") {
return ctx.Redirect("/")
}
types := []string{}
// // CPU
// cpuUsage := 0.0
// cpuUsages, err := cpu.Percent(1*time.Second, false)
for typeName := range arn.DB.Types() {
types = append(types, typeName)
// if err == nil {
// cpuUsage = cpuUsages[0]
// }
// // Memory
// memUsage := 0.0
// memInfo, _ := mem.VirtualMemory()
// if err == nil {
// memUsage = memInfo.UsedPercent
// }
// // Disk
// diskUsage := 0.0
// diskInfo, err := disk.Usage("/")
// if err == nil {
// diskUsage = diskInfo.UsedPercent
// }
// Host
platform, family, platformVersion, _ := host.PlatformInformation()
kernelVersion, _ := host.KernelVersion()
return ctx.HTML(components.Admin(user, platform, family, platformVersion, kernelVersion))
}
func average(floatSlice []float64) float64 {
if len(floatSlice) == 0 {
return 0
}
sort.Strings(types)
var sum float64
return ctx.HTML(components.Admin(user, types))
for _, value := range floatSlice {
sum += value
}
return sum / float64(len(floatSlice))
}

104
pages/admin/admin.pixy
View File

@ -1,31 +1,75 @@
component Admin(user *arn.User, types []string)
h2.page-title Admin Panel
component AdminTabs
.tabs
Tab("Server", "server", "/admin")
Tab("WebDev", "html5", "/admin/webdev")
Tab("Purchases", "shopping-cart", "/admin/purchases")
h3 Server
table
//- thead
//- tr
//- th Metric
//- th Value
tbody
tr
td CPU count:
td= runtime.NumCPU()
tr
td Goroutines:
td= runtime.NumGoroutine()
tr
td Go version:
td= runtime.Version()
h3 Types
table
//- thead
//- tr
//- th Table
tbody
each typeName in types
tr
td= typeName
td
a(href="/api/" + strings.ToLower(typeName) + "/")= "/api/" + strings.ToLower(typeName) + "/"
a.tab.ajax(href="/editor", aria-label="Editor")
Icon("pencil")
span.tab-text Editor
component Admin(user *arn.User, platform, family, platformVersion, kernelVersion string)
h1.page-title Admin Panel
AdminTabs
.admin
//- .widget.mountable
//- h3.widget-title Usage
//- table
//- tbody
//- tr
//- td CPU usage:
//- td
//- span= int(cpuUsage + 0.5)
//- span %
//- tr
//- td Memory usage:
//- td
//- span= int(memUsage + 0.5)
//- span %
//- tr
//- td Disk usage:
//- td
//- span= int(diskUsage + 0.5)
//- span %
.widget.mountable
h3.widget-title OS
table
tbody
tr
td Platform:
td= platform
tr
td Family:
td= family
tr
td Version:
td= platformVersion
tr
td Kernel:
td= kernelVersion
.widget.mountable
h3.widget-title Hardware
table
tbody
tr
td CPUs:
td= runtime.NumCPU()
.widget.mountable
h3.widget-title Go
table
tbody
tr
td Version:
td= runtime.Version()
tr
td Goroutines:
td= runtime.NumGoroutine()

36
pages/admin/purchases.go Normal file
View File

@ -0,0 +1,36 @@
package admin
import (
"net/http"
"sort"
"github.com/aerogo/aero"
"github.com/animenotifier/arn"
"github.com/animenotifier/notify.moe/components"
"github.com/animenotifier/notify.moe/utils"
)
// PurchaseHistory ...
func PurchaseHistory(ctx *aero.Context) string {
user := utils.GetUser(ctx)
if user == nil {
return ctx.Error(http.StatusUnauthorized, "Not logged in", nil)
}
if user.Role != "admin" {
return ctx.Error(http.StatusUnauthorized, "Not authorized", nil)
}
purchases, err := arn.AllPurchases()
if err != nil {
return ctx.Error(http.StatusInternalServerError, "Error fetching shop item data", err)
}
sort.Slice(purchases, func(i, j int) bool {
return purchases[i].Date > purchases[j].Date
})
return ctx.HTML(components.GlobalPurchaseHistory(purchases))
}

20
pages/admin/purchases.pixy Normal file
View File

@ -0,0 +1,20 @@
component GlobalPurchaseHistory(purchases []*arn.Purchase)
AdminTabs
h1.page-title All Purchases
table
thead
tr.mountable
th User
th Icon
th Item
th.history-quantity Quantity
th.history-price Price
th.history-date Date
tbody
each purchase in purchases
tr.shop-history-item.mountable(data-item-id=purchase.ItemID)
td
a.ajax(href=purchase.User().Link())= purchase.User().Nick
PurchaseInfo(purchase)

6
pages/webdev/webdev.go → pages/admin/webdev.go
View File

@ -1,9 +1,9 @@
package webdev
package admin
import "github.com/aerogo/aero"
import "github.com/animenotifier/notify.moe/components"
// Get ...
func Get(ctx *aero.Context) string {
// WebDev ...
func WebDev(ctx *aero.Context) string {
return ctx.HTML(components.WebDev())
}

48
pages/admin/webdev.pixy Normal file
View File

@ -0,0 +1,48 @@
component WebDev
AdminTabs
h1.page-title WebDev
.webdev
.widget.mountable
h3.widget-title Tests
.buttons
a.button.mountable(href="https://developers.google.com/speed/pagespeed/insights/?url=https://notify.moe/&tab=desktop", target="_blank", rel="noopener")
Icon("external-link")
span Google PageSpeed
a.button.mountable(href="https://observatory.mozilla.org/analyze.html?host=notify.moe", target="_blank", rel="noopener")
Icon("external-link")
span Mozilla Observatory
a.button.mountable(href="https://html5.validator.nu/?doc=https://notify.moe", target="_blank", rel="noopener")
Icon("external-link")
span HTML5 Validator
a.button.mountable(href="https://testmysite.withgoogle.com/", target="_blank", rel="noopener")
Icon("external-link")
span Mobile Speed
a.button.mountable(href="https://www.webpagetest.org/", target="_blank", rel="noopener")
Icon("external-link")
span Web Page Test
.widget.mountable
h3.widget-title Browser Support
.buttons
a.button.mountable(href="http://caniuse.com/#feat=webp", target="_blank", rel="noopener")
Icon("external-link")
span WebP
a.button.mountable(href="http://caniuse.com/#feat=push-api", target="_blank", rel="noopener")
Icon("external-link")
span Push API
a.button.mountable(href="http://caniuse.com/#feat=serviceworkers", target="_blank", rel="noopener")
Icon("external-link")
span Service Worker
a.button.mountable(href="http://caniuse.com/#feat=intersectionobserver", target="_blank", rel="noopener")
Icon("external-link")
span Intersection Observer
a.button.mountable(href="http://caniuse.com/#feat=requestidlecallback", target="_blank", rel="noopener")
Icon("external-link")
span Request Idle Callback
a.button.mountable(href="http://caniuse.com/#feat=css-variables", target="_blank", rel="noopener")
Icon("external-link")
span CSS Variables

21
pages/airing/airing.go
View File

@ -1,21 +0,0 @@
package airing
import (
"github.com/aerogo/aero"
"github.com/animenotifier/arn"
"github.com/animenotifier/notify.moe/components"
)
// Get ...
func Get(ctx *aero.Context) string {
var cache arn.ListOfIDs
err := arn.DB.GetObject("Cache", "airing anime", &cache)
airing, err := arn.GetAiringAnimeCached()
if err != nil {
return ctx.Error(500, "Couldn't fetch airing anime", err)
}
return ctx.HTML(components.Airing(airing))
}

3
pages/airing/airing.pixy
View File

@ -1,3 +0,0 @@
component Airing(animeList []*arn.Anime)
h2.page-title(title=toString(len(animeList)) + " anime") Airing
AnimeGrid(animeList)

Some files were not shown because too many files have changed in this diff Show More