wavesurfer

class: center, middle, inverse, title-slide

# wavesurfer
## Athos Petri Damiani
### School of Engineering - University of São Paulo
### rstudio::conf 2020

---

## About me and my work

.pull-left[

Athos Damiani, 31, Statistician from University of São Paulo, Brazil. Data scientist at Curso-R and R user since 2009. 
<hr/>
Currently working on my Master's on the theme:

<div align="center">
<strong style = "color: red">Automatic Bird Species Recognition Based on Their Songs</strong>
</div>

.small[
- Approach: Supervised Machine Learning
- Orientator: Prof. Doctor Linilson R. Padovese
- Co-orientator: Prof. Doctor Paulo Hubert
]
]

.pull-right[

![aplicacao_corujas](img/aplicacao_corujas3.png)

]

---

## Motivation

![aplicacao_corujas](img/abismo1.png)

---

## Motivation

![aplicacao_corujas](img/abismo2.png)

---

## wavesurfer.js

![](img/github_wavesurfer.png)

#### "A customizable audio waveform visualization, built on top of Web Audio API and HTML5 Canvas."

![](img/exemplos_wavesurfer.png)

---

## library(wavesurfer)

HTML widget for wavesurfer.js

```r
# shiny UI

wavesurfer("wavs_folder/wav_file.wav") %>% # or .mp3
* ws_annotator(labels = c("birdsong", "silence", "insect")) %>%
  ws_minimap() %>%
  ws_cursor()
```

![](img/saida_wavesurfer1.png)

---

## library(wavesurfer)

HTML widget for wavesurfer.js

```r
# shiny UI

wavesurfer(
  "wavs_folder/wav_file.wav", # or .mp3
* visualization = 'spectrogram'
) %>%
  ws_annotator(labels = c("birdsong", "silence", "insect")) %>% 
  ws_minimap() %>%
  ws_cursor()
```

![](img/saida_wavesurfer2.png)

---

## annotator_app()

Built-in shiny app for audio annotation.

.pull-left[

```r
annotator_app(
  "path/to/wav/files",
  "path/to/store/annotations"
)
```

### RDS layout (each row is a region)

<table class="table" style="font-size: 13px; width: auto !important; margin-left: auto; margin-right: auto;">
 <thead>
  <tr>
   <th style="text-align:left;"> audio_id </th>
   <th style="text-align:left;"> region_id </th>
   <th style="text-align:right;"> start </th>
   <th style="text-align:right;"> end </th>
   <th style="text-align:left;"> label </th>
  </tr>
 </thead>
<tbody>
  <tr>
   <td style="text-align:left;"> Glaucidium-minutissimum-13099.wav </td>
   <td style="text-align:left;"> 8ps588 </td>
   <td style="text-align:right;"> 1.9 </td>
   <td style="text-align:right;"> 3.0 </td>
   <td style="text-align:left;"> Glaucidium-minutissimum </td>
  </tr>
  <tr>
   <td style="text-align:left;"> Glaucidium-minutissimum-13099.wav </td>
   <td style="text-align:left;"> mm467 </td>
   <td style="text-align:right;"> 12.8 </td>
   <td style="text-align:right;"> 14.0 </td>
   <td style="text-align:left;"> Glaucidium-minutissimum </td>
  </tr>
  <tr>
   <td style="text-align:left;"> Glaucidium-minutissimum-13099.wav </td>
   <td style="text-align:left;"> 5t5hng </td>
   <td style="text-align:right;"> 18.8 </td>
   <td style="text-align:right;"> 19.9 </td>
   <td style="text-align:left;"> Glaucidium-minutissimum </td>
  </tr>
</tbody>
</table>

]

.pull-right[
![](img/folders.png)
]

---

## annotator_app()

![](img/annotator_app1.png)

---

## annotator_app()

![](img/annotator_app8.png)

---

## annotator_app()

![](img/annotator_app3.png)

---

## annotator_app()

![](img/annotator_app4.png)
---

## annotator_app()

![](img/annotator_app5.png)

---

## annotator_app()

![](img/annotator_app6.png)

---

## annotator_app()

![](img/annotator_app7.png)

---

## annotator_app()

pros

- Built in shiny for a better UX
- One can build a collaborative annotation app
- Two inputs (two folders)
- helper features like region suggestor (powered by `warbleR::auto_detec`)

cons

- X axis annotation only
- Need of shiny skills for fancier ideas

---
class: center, middle, inverse

## Thank You Very Much!

[<i class="fab fa-github"></i> `Athospd/wavesurfer`](http://github.com/Athospd/wavesurfer)

<i class="fab fa-twitter"></i> `@athos_damiani`

<i class="fa fa-envelope"></i> `athos_damiani@curso-r.com`