efim/scratch-screen-share
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
scratch-screen-share/notes.org
efim 73316581a0 still not getting data from buffer 
but the error is in gir::map-c-array
length is NIL, not sure what's up with that
2024-08-23 12:44:11 +00:00

635 lines
25 KiB
Org Mode
Raw Blame History

* [2024-06-28 Fri]
** well, i'm setting up sbcl and sly?
because i imagine using Common Lisp for my attempt at building screen
sharing for the XR glasses
for the cool runtime access, right?
** well, just flake shell and (use-package sly) is enough
for .lisp files to get stuff
yes, i'm still using C-M-f and stuff to move around sexps but things are fine
so, what's next?
well, this is nice high level description
dbus
https://www.freedesktop.org/wiki/Software/dbus/
and here's something to start usage
https://blog.macrolet.net/posts/DBus-and-PolicyKit-from-Common-Lisp.html
from https://github.com/death/dbus
and even lots of examples, but i think i will need a big book on Common Lisp to learn new things?
* [2024-06-29 Sat]
** going through the dbus guide, sounds neat
https://develop.kde.org/docs/features/d-bus/introduction_to_dbus/
that qdbus and qdbus-viewer are in
#+begin_src bash
$ nix shell nixpkgs#kdePackages.qttools
#+end_src
** now let's read about calling methods
* the receiving of the signals
* all the different tabs i had
** overall freedesktop portal dbus api docs
https://docs.flatpak.org/en/latest/portal-api-reference.html#gdbus-org.freedesktop.portal.ScreenCast
** searching github for 'get-managed-objects usage
https://github.com/search?q=get-managed-objects&type=code
** searchin github for 'define-singlan-handler usage examples
https://github.com/search?q=dbus%3Adefine-dbus-signal-handler&type=code
** example in cl-notify of calling a method 'with-introspected-object
https://github.com/Lautaro-Garcia/cl-notify/blob/main/src/dbus-protocol.lisp#L27
#+begin_src lisp
(defmacro with-dbus-method-inovaction ((result-var method &rest args) &body body)
(alexandria:with-gensyms (bus notifications-object)
`(dbus:with-open-bus (,bus (dbus:session-server-addresses))
(dbus:with-introspected-object (,notifications-object ,bus "/org/freedesktop/Notifications" "org.freedesktop.Notifications")
(let ((,result-var (,notifications-object "org.freedesktop.Notifications" ,method ,@args)))
,@body)))))
#+end_src
** example in cl-notify of working with signals
https://github.com/Lautaro-Garcia/cl-notify/blob/main/src/signals.lisp
they register single handler, and add callbacks dynamically & call
them from the registered function
** dbus library example for publishing objects
https://github.com/death/dbus/blob/master/examples/publish.lisp
** my request for help with interactive async calls
https://github.com/death/dbus/issues/31
** old ticket on dbus repo about enabling introspection
https://github.com/death/dbus/issues/23
** example of signal-handler in stumpwm-thingy
https://github.com/lokedhs/stumpwm-dbus/blob/9cf0b52876111e777da27a42e3c81269b97c0005/src/pidgin.lisp#L16
** ref for symbold in dbus package
https://quickref.common-lisp.net/dbus.html#The-dbus_002fconnections-system
** blog article with basics of usage
https://blog.macrolet.net/posts/DBus-and-PolicyKit-from-Common-Lisp.html
** big article about iolib
https://pages.cs.wisc.edu/~psilord/blog/data/iolib-tutorial/tutorial.html
it's the one that's used to create multiplexer for communication
i tried to figure out what it means to have event-loops
** docs on dbus Screenshots portal
https://flatpak.github.io/xdg-desktop-portal/docs/doc-org.freedesktop.portal.Screenshot.html
the methods and stuff
** docs on dbus type system
https://dbus.freedesktop.org/doc/dbus-specification.html#type-system
i.e how to understand types of methods and properties
#+begin_src lisp
CL-USER> (dbus:sigexp "xsaysas")
(:INT64 :STRING (:ARRAY :BYTE) :STRING (:ARRAY :STRING))
#+end_src
function signature explanation can show what should lisp types be
https://github.com/death/dbus/issues/17#issuecomment-418920440
** example of flameshot how they get Screenshot via dbus
https://github.com/flameshot-org/flameshot/blob/c1dac52231024174faa68a29577129ebca125dff/src/utils/screengrabber.cpp#L59
** dbus methods for Notification
https://specifications.freedesktop.org/notification-spec/notification-spec-latest.html#signals
** dbus methods of ScreenCast
https://flatpak.github.io/xdg-desktop-portal/docs/doc-org.freedesktop.portal.ScreenCast.html
how to start them etc
** dbus spec on Request & Response for the async methods
https://flatpak.github.io/xdg-desktop-portal/docs/doc-org.freedesktop.portal.Request.html#org-freedesktop-portal-request
** repo for xdg-portal for wayland
https://github.com/emersion/xdg-desktop-portal-wlr
** the python snippet to check that ScreenCast via xdg-portal works
https://gitlab.gnome.org/-/snippets/19
* [2024-07-20 Sat]
** and now i want to try to get display of stuff into gtk window, i guess
** doc about cl bindings to gtk3
https://www.crategus.com/books/cl-gtk/gtk-tutorial.html#example-window-simple-c
** well, i guess i could go and try to read guide on pipewire with C examples
https://docs.pipewire.org/page_tutorial.html
this is tutorial for the system
and here is example program that maybe does similar thing
https://docs.pipewire.org/video-play_8c-example.html
* [2024-07-28 Sun]
** ok, now i'm trying to get frames from the pipewire 'out' node
and one option i'm exploring is gstreamer
which is a framework for creating applications that stream video / audio
with some kind of ability to define \ declare pipeline and then start
and then use api somehow
there's c api, and gobject accessibility layer
so either c to common lisp, or gobject common lisp thingy maybe exist
** reading docs
https://gstreamer.freedesktop.org/documentation/frequently-asked-questions/using.html?gi-language=c
added via
gst_all_1.gstreamer
** more readin
gst-launch-1.0 videotestsrc ! videoconvert ! autovideosink
this is test for how it's installed
needed to add plugins
https://nixos.wiki/wiki/GStreamer
** i guess i can try to go with the tutorials?
https://gstreamer.freedesktop.org/documentation/tutorials/index.html?gi-language=c
but with common lisp over gobject?
i think i'm already using this in the gtk bindings playground? maybe not
https://cl-gtk2.common-lisp.dev/doc/gobject/Creating-GObjects-classes-and-implementing-GInterfaces.html
#+begin_quote
GObject binding at the moment provides only limited scenarios of creating GObject classes.
#+end_quote
** and there's something
https://github.com/BohongHuang/cl-gobject-introspection-wrapper
which is based on older
https://github.com/andy128k/cl-gobject-introspection
https://search.nixos.org/packages?channel=unstable&from=0&size=50&sort=relevance&type=packages&query=gobject
they are packaged in nix, this is just unbelievable how much is done by humanity, omg
* [2024-08-15 Thu]
** getting back after a long while.
so, i was at the creating a window, not knowing how to stream video from pipewire out node
so. what to do, huh
i think gtk3 bindings library was separate from gobject and glib
but maybe it doesn't matter?
or maybe i should figure out glib bindings and use gtk3 though it?
** well, there's something simple with opening a window
[[file:gtk-playground.lisp::defun example-window-simple (]]
** and something extra simple with gstreamer opening separate window for test video source
[[file:gstreamer/tutorial-hello-world.lisp]]
** maybe try to open gstreamer window with screen share?
ok, when i try example of pipewire window on screenshare source
i get error that pipewiresrc is not found
and that's right, i used system-wide gst-launch-1.0
and when i tried with one in flake shell - it also didn't work
so. tried removing all from shell env, then not found namespace Gst
(defvar *gstreamer* (gir:require-namespace "Gst"))
so. yeah. maybe i need to install dev headers or something?
** so. for moving the video around. chatgpt recommended something named 'videobox'
like i can declare it in the pipeline and then get reference to it and use to set video coordinates, huh?
https://gstreamer.freedesktop.org/documentation/videobox/index.html?gi-language=python
the thing exists but i have zero idea whether it's accessible in a way that's recommended
#+begin_src lisp
(defun for-screencast ()
(gir:invoke (*gstreamer* 'init) '())
(let* ((pipeline (gir:invoke (*gstreamer* 'parse-launch)
"pipewiresrc path=49 ! videoconvert ! videobox top=0 left=0 right=0 bottom=0 ! videoconvert ! autovideosink"
))
(videobox (gir:invoke (pipeline 'get-element-by-name) "videobox")))
;; Set the initial state to playing
(gir:invoke (pipeline 'set-state) (gir:nget *gstreamer* "State" :playing))
;; Function to move the video
(defun move-video (top left)
(gir:invoke (videobox 'set-property) 'top top)
(gir:invoke (videobox 'set-property) 'left left))
;; Example: Move the video 10 pixels down and 20 pixels to the right
(move-video 10 20)))
#+end_src
and similar advice from claude, huh
#+begin_src lisp
(defun for-screencast ()
(gir:invoke (*gstreamer* 'init) '())
(let* ((pipeline (gir:invoke (*gstreamer* 'parse-launch)
"pipewiresrc path=49 ! videoconvert ! videobox name=move ! videoscale ! autovideosink"))
(move-element (gir:invoke (pipeline 'get-by-name) "move")))
(gir:invoke (pipeline 'set-state) (gir:nget *gstreamer* "State" :playing))
(values pipeline move-element)))
(defun move-video (move-element left top)
(gir:set-property move-element "left" left)
(gir:set-property move-element "top" top))
;; Usage:
(multiple-value-bind (pipeline move-element) (for-screencast)
;; Move video 50 pixels to the right and 30 pixels down
(move-video move-element 50 30)
;; Your main loop or other logic here
)
#+end_src
let's try this out
* [2024-08-16 Fri]
** new day of the vacation
the experiment with moving the video, right?
could it be that both examples are fake? i don't seem to find both 'get-element things
trying to find anything about dynamism in gstreamer
https://coaxion.net/blog/2014/01/gstreamer-dynamic-pipelines/
looks like this is about changing elements of the pipeline
and it's possible that parameters of the pipeline elements (pads?) are also static
** options for transforming frames?
*** videotransform : A combination of scaling, 90-degree-step rotation,
https://github.com/Freescale/gstreamer-imx/blob/master/README.md
horizontal/vertical flipping, and color space conversion
operations. These elements are able to render video overlay
compositions from GstVideoOverlayCompositionMeta data. They also
implement the GstVideoDirection interface and have a "video-direction"
property that handles rotation and flipping. Through this property, it
is possible to configure these elements to auto-rotate images
according to the information in image-orientation tags.
*** glimagesink
https://gstreamer.freedesktop.org/documentation/opengl/glimagesink.html?gi-language=python
gst-launch-1.0 -v videotestsrc ! video/x-raw ! glimagesink
has error
ERROR: from element /GstPipeline:pipeline0/GstGLImageSinkBin:glimagesinkbin0/GstGLImageSink:sink: Failed to create EGLDisplay from native display
Additional debug info:
../ext/gl/gstglimagesink.c(1136): _ensure_gl_setup (): /GstPipeline:pipeline0/GstGLImageSinkBin:glimagesinkbin0/GstGLImageSink:sink
ERROR: pipeline doesn't want to preroll.
so maybe opengl is not available? how to check?
*** gltransformation
https://gstreamer.freedesktop.org/documentation/opengl/gltransformation.html?gi-language=python
here examples seem to be for images, not video
and same error, let's figure out the opengl check
** somewhat categorized things?
* [2024-08-21 Wed]
** visiting sister, returning back
last time i got stuck on video displaying frozen when i tried to add appsink
just re-checking my basic gstreamer pipeline and screencast start - thing works
but i want to clean it up i guess, to allow for simpler retries?
** so, for the appsnk there are signals
https://gstreamer.freedesktop.org/documentation/app/appsink.html?gi-language=python#appsink::new-sample
new-sample
#+begin_src python
def new_sample_callback (appsink, udata):
#python callback for the 'new-sample' signal
#+end_src
Signal that a new sample is available.
but how do i connect to a callback from common lisp and gobject introspection
https://github.com/andy128k/cl-gobject-introspection
does the gitlab repo has info on callbacks?
can't seem to call method 'pull-sample'
maybe article will help?
https://gstreamer.freedesktop.org/documentation/application-development/advanced/pipeline-manipulation.html?gi-language=python#grabbing-data-with-appsink
well, this looks like method on gst namespace? "gst_app_sink_pull_sample()"
https://gstreamer.freedesktop.org/documentation/applib/gstappsink.html?gi-language=python#gst_app_sink_pull_sample
there's article for python bindings
https://lifestyletransfer.com/how-to-use-gstreamer-appsink-in-python/
* [2024-08-22 Thu]
** hey, i think this is progress
i am able to call appsink 'pull-sample after all (for some reason, i don't know what i was doing differently)
new error:
>>> got to process sample with #<STRUCT-INSTANCE {10128D02F3}>
but in the code for processing of the sample some type errors
Backtrace:
0: ((:METHOD GIR::THIS-OF (T)) T) [fast-method]
1: (GIR:PROPERTY T DATA)
2: (PROCESS-FRAME #<GIR::STRUCT-INSTANCE {100BCDAB73}>)
3: ((LAMBDA (APPSINK) :IN START-PROCESSING) #<GIR::OBJECT-INSTANCE {100BBE02D3}>)
ok, but when i have struct instance, how do i introspect it?
** let's just go and read documentation for the type that's returned from pull-sample, sure
also there's that python code, right?
doc 1: get sample on appsink https://people.freedesktop.org/~tsaunier/documentation/gst-plugins-base-app-1.0/gstappsink.html?gi-language=c#gst_app_sink_pull_sample
doc 2: GstSample class https://people.freedesktop.org/~tsaunier/documentation/gstreamer-1.0/gstsample.html?gi-language=c#GstSample
so on GstSample everything is taken as results of funciton calls? let's try that
well, maybe i don't need get-info on sample, it returns null, maybe i need it on buffer?
doc 3: get-buffer on sample https://people.freedesktop.org/~tsaunier/documentation/gstreamer-1.0/gstsample.html?gi-language=python#gst_sample_get_buffer
doc 4: buffer class info https://people.freedesktop.org/~tsaunier/documentation/gstreamer-1.0/gstbuffer.html?gi-language=python#gst_buffer_peek_memory
** i'm recommended to look into GstBuffer.map
https://people.freedesktop.org/~tsaunier/documentation/gstreamer-1.0/gstbuffer.html?gi-language=python#gst_buffer_map
it returns some https://people.freedesktop.org/~tsaunier/documentation/gstreamer-1.0/gstmemory.html?gi-language=python#GstMapInfo
** struggling, searching for answers
https://github.com/andy128k/cl-gobject-introspection/issues/73#issuecomment-755316735
Examples of inspection of a repository can be found in test files.
https://github.com/andy128k/cl-gobject-introspection/blob/master/test/test1.lisp
and i mean it's possible that referencing members of a structure is not implemented?
so, huh
using internal functions
#+begin_src lisp
(data (gir:fields-dict-of (gir:struct-class-of map-info)))
#+end_src
i see
>>>> and data has size NIL and buf ((memory . #<FIELD-INFO {100FEB4083}>)
(flags . #<FIELD-INFO {100FEB4153}>)
(data . #<FIELD-INFO {100FEB4223}>)
(size . #<FIELD-INFO {100FEB42F3}>)
(maxsize . #<FIELD-INFO {100FEB43C3}>)
(user_data . #<FIELD-INFO {100FEB4493}>)
(_gst_reserved
. #<FIELD-INFO {100FEB4563}>))
; Evaluation aborted on T
CAPTURING-FRAMES>
so. there are data and size fields.
maybe i can try with
** well, for some reason data is not avaiable?
(let* ((buffer (gir:invoke (sample 'get-buffer)))
(info (gir:invoke (sample 'get-info)))
(caps (gir:invoke (sample 'get-caps)))
(caps-struct (gir:invoke (caps 'get-structure) 0))
(width (gir:invoke ((gir:invoke (caps-struct 'get-value) "width") 'get-int)))
(height (gir:invoke ((gir:invoke (caps-struct 'get-value) "height") 'get-int)))
(buf-size (gir:invoke (buffer 'get-size)))
(data-copy (gir:invoke (buffer 'extract-dup) 0 buf-size)))
(format t ">>>> at buffer got size ~A and tried to copy ~A~%" buf-size data-copy)
;; ok, we have buffer, let's try to get copy
;; (multiple-value-bind (found map-info) (gir:invoke (buffer 'map) (gir:nget *gstreamer* "MapFlags" :read))
;; (format t ">>>> the buffer has offset ~A~%" (gir:field buffer 'offset))
;; (format t ">>>> what's in struct? ~A~%" (gir:invoke (caps-struct 'to-string)))
;; (format t ">>>> the fields are width ~A, height ~A~%" width height)
;; (format t ">>>> and map result are ~A and ~A~%" found map-info)
;; (when found
;; (let ((data (gir:field map-info 'memory))
;; (size (gir:field map-info 'size)))
;; (format t ">>>> and struct info? ~A~%" map-info)
;; (gir:invoke (buffer 'unmap) map-info)
;; (format t ">>>> and data has size ~A and memory is ~A~%" size (or data "empty data")))))
)
data-copy is NIL
yep. mda.
** well, trying to copy as in python example didn't work either
(buf-size (gir:invoke (buffer 'get-size)))
(data-copy (gir:invoke (buffer 'extract-dup) 0 buf-size))
https://github.com/jackersson/gst-python-tutorials/blob/master/launch_pipeline/run_appsink.py#L49
yeah, it's possible that gir library doesn't work well? even if on c level all is ok?
* [2024-08-23 Fri]
** well, problem is in trying to read gir:map-c-array
it gets length nil and fails
GIR::MAP-C-ARRAY
so what i'm using:
https://github.com/andy128k/cl-gobject-introspection
and there's wrapper around it
https://github.com/bohonghuang/cl-gobject-introspection-wrapper
but there is also
https://github.com/kat-co/gir2cl
maybe it will work?
or maybe i'll figure out what's up
** well, trying to check available slots on that array type
(mapcar #'c2mop:slot-definition-name (c2mop:class-slots (class-of c-array-type)))
=> (PARAM-TYPE FIXED-SIZE ZERO-TERMINATED? LENGTH)
and how do i get their values? if i don't know name of the :reader ?
( from article on https://riptutorial.com/common-lisp/example/9826/obtain-the-slot-names-of-a-class )
but yeah. huh.
** let's write up the problem?
I'm trying to use gstreamer - a framework for video\audio pipelines with gobject api
the hope is to use their 'appsink' to get video frame data into the program i'm writing
https://people.freedesktop.org/~tsaunier/documentation/gstreamer-1.0/gstbuffer.html?gi-language=python#gst_buffer_map
i'll link the code in case it will be needed for replication,
right now it seems that by doing gir calls i've successfully set up pipeline
and receiving callbacks with the buffers
the problem i get is when i try to access 'data
and it seems to come from GIR:MAP-C-ARRAY
where length of the parameter c-array-type is NIL
in #'map-c-array it comes by calling
#+begin_src lisp
(defun c-array-length (c-array-type)
(with-slots (param-type fixed-size zero-terminated? length)
c-array-type
(if fixed-size
(if (and length (< length fixed-size))
length
fixed-size)
(if length
(+ length (if zero-terminated? 1 0))))))
#+end_src
and manually checking slots of the 'c-array-type
it has NIL in length
right now I don't know whether it's a problem on the side of cl-gobject-introspection
in how it's initializing the c-array-type
or maybe it's somehow problem on the side of the gstreamer library,
that provides bad responses, i'll try to use other languages \ libraries
but could also really use some help in figuring this out
fuller log:
#+begin_src
The value
NIL
is not of type
REAL
[Condition of type TYPE-ERROR]
Restarts:
0: [ABORT] abort thread (#<FOREIGN-THREAD tid=127655 "callback" RUNNING {10131D8033}>)
Backtrace:
0: (SB-KERNEL:TWO-ARG->= 0 NIL)
1: (GIR::MAP-C-ARRAY #<FUNCTION (LAMBDA (GIR::POS) :IN GIR::MEM-GET) {10131D9CBB}> #.(SB-SYS:INT-SAP #X7F1EC80EBBD8) #<GIR::C-ARRAY-TYPE {10131E0FA3}>)
Locals:
ARRAY = #.(SB-SYS:INT-SAP #X7F1EC80EBBD8)
C-ARRAY-TYPE = #<GIR::C-ARRAY-TYPE {10131E0FA3}>
FUNC = #<FUNCTION (LAMBDA (GIR::POS) :IN GIR::MEM-GET) {10131D9CBB}>
I = 0
LENGTH = NIL
PARAM-SIZE = 1
POS = #.(SB-SYS:INT-SAP #X7F1EC80EBBD8)
2: ((:METHOD GIR::MEM-GET (T GIR::POINTER-TYPE)) #.(SB-SYS:INT-SAP #X7F1F97F7FFF8) #<GIR::POINTER-TYPE {10131E0FF3}>) [fast-method]
Locals:
GIR::POS = #.(SB-SYS:INT-SAP #X7F1F97F7FFF8)
TYPE = #<GIR::POINTER-TYPE {10131E0FF3}>
3: (GIR.FIELD:GET #.(SB-SYS:INT-SAP #X7F1ECC0036F0) #<GIR:FIELD-INFO {10087D1133}>)
4: (PROCESS-FRAME #<GIR::STRUCT-INSTANCE {10131E02D3}>)
5: ((LAMBDA (APPSINK) :IN START-PROCESSING) #<GIR::OBJECT-INSTANCE {10131E01B3}>)
6: ((LAMBDA (GIR::CLOSURE RETURN GIR::N-VALUES GIR::PARAMS GIR::HINT GIR::DATA) :IN "/nix/store/hijk9y2lcv24s3a5kv56cyxqcn3frgs4-source-patched/src/signal.lisp") #.(SB-SYS:INT-SAP #X7F1FCC470B30) #.(SB-S..
7: ((LAMBDA (SB-ALIEN::ARGS-POINTER SB-ALIEN::RESULT-POINTER FUNCTION) :IN "/nix/store/hijk9y2lcv24s3a5kv56cyxqcn3frgs4-source-patched/src/signal.lisp") #<unavailable argument> #<unavailable argument> #<..
8: ((FLET SB-UNIX::BODY :IN SB-THREAD::RUN))
9: ((FLET "WITHOUT-INTERRUPTS-BODY-" :IN SB-THREAD::RUN))
10: ((FLET SB-UNIX::BODY :IN SB-THREAD::RUN))
11: ((FLET "WITHOUT-INTERRUPTS-BODY-" :IN SB-THREAD::RUN))
12: (SB-THREAD::RUN)
13: (SB-THREAD::ENTER-FOREIGN-CALLBACK 87 69885224809132 69885224809124)
14: ("foreign function: call_into_lisp_")
15: ("foreign function: funcall3")
16: ("foreign function: callback_wrapper_trampoline")
17: ("foreign function: #x501053AB")
--more--
#+end_src
the source:
#+begin_src lisp
;(load (sb-ext:posix-getenv "ASDF"))
(asdf:load-system 'cl-cffi-gtk)
(asdf:load-system 'cl-gobject-introspection)
(defpackage :capturing-frames
(:use :cl))
(in-package :capturing-frames)
(defvar *gstreamer* (gir:require-namespace "Gst"))
;(defvar *gstapp* (gir:require-namespace "GstApp"))
(defun cast-with-videobox (&optional stream-num)
(gir:invoke (*gstreamer* 'init) '())
(let* ((pipeline (gir:invoke (*gstreamer* "Pipeline" 'new) "video-pipeline"))
(source (gir:invoke (*gstreamer* "ElementFactory" 'make) (if stream-num
"pipewiresrc"
"videotestsrc") "source"))
(convert (gir:invoke (*gstreamer* "ElementFactory" 'make) "videoconvert" "convert"))
(tee (gir:invoke (*gstreamer* "ElementFactory" 'make) "tee" "tee"))
(queue1 (gir:invoke (*gstreamer* "ElementFactory" 'make) "queue" "queue1"))
(queue2 (gir:invoke (*gstreamer* "ElementFactory" 'make) "queue" "queue2"))
(sink (gir:invoke (*gstreamer* "ElementFactory" 'make) "autovideosink" "sink"))
(appsink (gir:invoke (*gstreamer* "ElementFactory" 'make) "appsink" "appsink"))
)
(when stream-num
;; Set pipewiresrc properties, such as the path
(setf (gir:property source 'path) (format nil "~A" stream-num)))
;; Add and link elements in the pipeline
(gir:invoke (pipeline 'add) source)
(gir:invoke (pipeline 'add) convert)
(gir:invoke (pipeline 'add) tee)
(gir:invoke (pipeline 'add) queue1)
(gir:invoke (pipeline 'add) queue2)
(gir:invoke (pipeline 'add) sink)
(gir:invoke (pipeline 'add) appsink)
(gir:invoke (source 'link) convert)
(gir:invoke (convert 'link) tee)
(gir:invoke (tee 'link) queue1)
(gir:invoke (tee 'link) queue2)
(gir:invoke (queue1 'link) sink)
(gir:invoke (queue2 'link) appsink)
;; Configure appsink
(setf (gir:property appsink "emit-signals") t
(gir:property appsink "max-buffers") 1
(gir:property appsink "drop") t)
(gir:invoke (pipeline 'set-state) (gir:nget *gstreamer* "State" :playing))
(values pipeline appsink)))
(defun process-frame (sample)
(format t ">>> got to process sample with ~A~%" sample)
(let* ((buffer (gir:invoke (sample 'get-buffer)))
(info (gir:invoke (sample 'get-info)))
(caps (gir:invoke (sample 'get-caps)))
(caps-struct (gir:invoke (caps 'get-structure) 0))
(width (gir:invoke ((gir:invoke (caps-struct 'get-value) "width") 'get-int)))
(height (gir:invoke ((gir:invoke (caps-struct 'get-value) "height") 'get-int))))
(multiple-value-bind (found map-info) (gir:invoke (buffer 'map) (gir:nget *gstreamer* "MapFlags" :read))
(format t ">>>> the buffer has offset ~A~%" (gir:field buffer 'offset))
(format t ">>>> what's in struct? ~A~%" (gir:invoke (caps-struct 'to-string)))
(format t ">>>> the fields are width ~A, height ~A~%" width height)
(format t ">>>> and map result are ~A and ~A~%" found map-info)
(when found
(let ((data (gir:field map-info 'data))
(size (gir:field map-info 'size)))
(format t ">>>> and struct info? ~A~%" map-info)
(gir:invoke (buffer 'unmap) map-info)
(format t ">>>> and data has size ~A and data is ~A~%" size (or data "empty data")))))))
(defun start-processing (appsink)
(gir:connect appsink "new-sample"
(lambda (appsink)
(format t ">> got printed object ~A~%" appsink)
(format t ">> got printed class ~A~%" (gir:gir-class-of appsink))
(format t ">> trying to get method on sink ~A~%" (gir:nget appsink 'pull-sample))
(let ((sample (gir:invoke (appsink 'pull-sample))))
(when sample
(process-frame sample)))
(gir:nget *gstreamer* "FlowReturn" :ok))))
(defun try-run ()
(multiple-value-bind (pipeline appsink) (cast-with-videobox)
(format t "got appsink ~A" appsink)
(start-processing appsink)
(loop
(sleep 1))))
;(try-run)
#+end_src
Reference in New Issue View Git Blame Copy Permalink